Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note4 min read

GIF: Locally Sound Geometric Information Flow Control for LLMs

Source

arXiv

arxiv.org/abs/2606.23277v1

Read the source

Source

Plain-Language Brief

Researchers have proposed a new method called Geometric Information Flow (GIF) for controlling information flow in large language models used in agentic systems. GIF uses the model's Jacobian and local output geometry to upper-bound the Shannon mutual information between perturbed input spans and model outputs, providing a scalable measure that can be computed on large models. The method outperforms attention-based baselines in integrity and confidentiality tasks, using less token cost while still achieving high performance.

What the Document Says

  • The paper presents GIF (Geometric Information Flow), a semantic framework for tracking information flow from input tokens to outputs in large language models.
  • GIF uses the model's Jacobian and local output geometry to upper-bound the Shannon mutual information between perturbed input spans and model outputs.
  • The method is scalable and can be computed on large models via automatic differentiation and low-rank approximation.
  • Unlike attention-based or correlational attribution heuristics, GIF satisfies local geometric soundness.
  • A Lean 4 proof has been provided for the method showing that it upper-bounds the true information flow induced by a given prompt under local regularity assumptions.
  • The researchers evaluated GIF on various integrity and confidentiality tasks across multiple prompt-injection and privacy-leakage benchmarks.
  • GIF achieved near-perfect recall even without a downstream declassifier, outperforming attention-based baselines.
  • Combined with lightweight LLM-based declassifiers, GIF matches or exceeds the F1 of direct LLM-as-judge baselines such as GPT-5.5.
  • GIF uses up to 81x lower token cost compared to direct LLM-as-judge baselines.
  • The flows detected with small surrogate models transfer to larger state-of-the-art models and other model families, even when the surrogate is up to 200x smaller.

Key Concepts Mentioned

  • Large Language Models
  • Information Flow Control
  • Geometric Information Flow (GIF)
  • Shannon mutual information
  • Jacobian
  • Local output geometry
  • Integrity and confidentiality tasks
  • Prompt-injection benchmarks
  • Privacy-leakage benchmarks
  • Lean 4 proof
  • Automatic differentiation
  • Low-rank approximation
  • LLM-based declassifiers
  • GPT-5.5

Claims Made by the Document

  • GIF is a semantic framework for tracking information flow in large language models.
  • GIF upper-bounds the true information flow induced by a given prompt under local regularity assumptions.
  • GIF outperforms attention-based baselines in integrity and confidentiality tasks.
  • Combined with lightweight LLM-based declassifiers, GIF matches or exceeds the F1 of direct LLM-as-judge baselines such as GPT-5.5.
  • GIF uses less token cost compared to direct LLM-as-judge baselines.

Evidence Used

  • The paper describes in detail how GIF is designed and implemented.
  • The authors provide a Lean 4 proof for the method's upper-bounding of information flow.
  • Evaluation results are presented showing GIF's performance in integrity and confidentiality tasks.
  • Comparisons with attention-based baselines are provided.

Methods / Approach

  • Geometric Information Flow (GIF)
  • Lean 4 proof
  • Automatic differentiation
  • Low-rank approximation

Limitations Stated by the Authors

  • The authors mention that existing approaches for information flow control suffer from severe taint explosion.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • It is unknown if GIF can handle unforeseen complexities or attacks not considered in the research.
  • It is also unknown how well GIF performs on tasks outside of integrity and confidentiality, or on smaller language models.

Open Questions

  • How does GIF perform on tasks outside of integrity and confidentiality?
  • Can GIF handle unforeseen complexities or attacks not considered in the research?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The paper focuses on information flow control for large language models, particularly addressing security and privacy risks associated with their use.

Raw Abstract / Excerpt

Large language models increasingly mediate interactions between sensitive data, untrusted inputs, and privileged actions in agentic systems, creating security and privacy risks. These range from prompt injections that manipulate downstream tool use to leakage of confidential information through model outputs. Recent Information Flow Control (IFC)-based defenses show promise but lack a principled semantic foundation for reasoning about information flow through the model itself. Since any input token may influence any output token in an autoregressive LLM, existing approaches suffer from severe taint explosion. We present Geometric Information Flow (GIF), a semantic framework for tracking information flow from input tokens to outputs. GIF uses the LLM Jacobian and local output geometry to upper-bound the Shannon mutual information between perturbed input spans and model outputs, yielding a scalable measure computable on large models via automatic differentiation and low-rank approximation. Unlike attention-based or correlational attribution heuristics, GIF satisfies local geometric soundness, and we provide a fully mechanized Lean 4 proof that it upper-bounds the true information flow induced by a given prompt under local regularity assumptions. We evaluate GIF on integrity and confidentiality tasks across multiple prompt-injection and privacy-leakage benchmarks. GIF achieves near-perfect recall even without a downstream declassifier, outperforming attention-based baselines. Combined with lightweight LLM-based declassifiers, it matches or exceeds the F1 of direct LLM-as-judge baselines such as GPT-5.5 xhigh reasoning while using up to 81x lower token cost. GIF flows detected with small surrogate models transfer to larger state-of-the-art models and other model families, even when the surrogate is up to 200x smaller, suggesting black-box deployment without gradient access.