Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Bole Ma, Jan Eitzinger, Harald Koestler, Gerhard Wellein
• Venue: arXiv: cs.DC
• Published: 2026-06-22
• Collected: 2026-06-23
• URL: http://arxiv.org/abs/2606.23581v1
• PDF: https://arxiv.org/pdf/2606.23581v1
• DOI: UNKNOWN

Plain-Language Brief

The paper presents Kamera, a unified position-invariant multimodal KV cache that allows for training-free reuse of cached data in a way that avoids the need to recompute every look-back from scratch. The proposed solution addresses the failure mode prior position-independent caches have when dealing with multi-hop reasoning, by introducing a small low-rank conditioning patch stored alongside each position-free chunk.

What the Document Says

• The paper presents a unified position-invariant multimodal KV cache called Kamera.
• Kamera allows for training-free reuse of cached data, avoiding the need to recompute every look-back from scratch.
• In order to address the failure mode prior position-independent caches have when dealing with multi-hop reasoning, a small low-rank conditioning patch is introduced and stored alongside each position-free chunk.
• The proposed solution makes three window operations cheap: reorder, sliding-window survival, and recall.
• A rank-m patch recovers full task accuracy on cross-chunk-binding benchmarks.
• The conditioning signal is strongest in redundant vision and video streams.

Key Concepts Mentioned

• Multimodal agents
• KV cache
• position-invariant
• reuse
• recompute
• low-rank conditioning patch

Claims Made by the Document

• Kamera allows for training-free reuse of cached data without the need to recompute every look-back from scratch.
• The proposed solution addresses the failure mode prior position-independent caches have when dealing with multi-hop reasoning.
• A rank-m patch recovers full task accuracy on cross-chunk-binding benchmarks.

Evidence Used

• The paper describes a new KV cache called Kamera that allows for training-free reuse of cached data without the need to recompute every look-back from scratch.
• The paper identifies the failure mode prior position-independent caches have when dealing with multi-hop reasoning, and proposes a solution using a low-rank conditioning patch.

Methods / Approach

• Introducing a small low-rank conditioning patch stored alongside each position-free chunk.
• Three window operations: reorder, sliding-window survival, and recall.

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

The effectiveness of Kamera in other domains or with different types of agents is not discussed in the paper.

Open Questions

• How does the performance of Kamera compare to other caching strategies in various scenarios?
• What are the potential drawbacks of using low-rank conditioning patches?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper focuses on the use of Kamera for multimodal agents, specifically addressing the failure mode prior position-independent caches have when dealing with multi-hop reasoning. The proposed solution makes three window operations cheap: reorder, sliding-window survival, and recall.

Raw Abstract / Excerpt

Multimodal agents repeatedly re-examine the same video frames, UI screenshots, and rendered artifacts as their context window slides and reasoning iterates, yet every look-back re-encodes from scratch, because prefix caches serve reuse only at a fixed leading position. We show this recompute is avoidable, and identify exactly what naive KV reuse loses: the cross-chunk conditioning a chunk absorbs from its neighbours. This loss is asymmetric. The direct readout of a cached chunk is recovered exactly and for free by the standard state-merge. What remains is a diffuse, low-rank residue concentrated in deep layers, invisible to single-hop retrieval but precisely what multi-hop reasoning binds on. Blind reuse therefore leaves single-hop recall intact while halving multi-hop accuracy; this is the failure mode prior position-independent caches, designed for single-context or single-image reuse, do not address. We repair it with a small, training-free low-rank conditioning patch stored alongside each position-free chunk. Reuse reduces to one operator across MLA, GQA, and MHA: exact RoPE re-rotation to any target position, plus the patch that restores cross-chunk binding. This makes three window operations cheap: reorder (one patch serves every ordering of a cached set), sliding-window survival (surviving chunks relocate via rotation only, zero re-encode), and recall (an evicted chunk is rehydrated by its patch, never re-encoded). A rank-m patch recovers full task accuracy on cross-chunk-binding benchmarks, MM-NIAH across two attention families and two-page doc-QA, at a fraction of the KV footprint, and reconstructs re-prefill KV to within bf16 rounding in a production SGLang kernel across six backbones. The conditioning signal is strongest in redundant vision and video streams, making our solution most impactful where multimodal agents spend their recompute budget.