FIELD NOTES TOPIC
cache compression
2 notes, newest first.
DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression
The research introduces DepthWeave-KV, a new cache compression method for long-context language models. This method adapts to the needs of different tokens by factorizing key and value states across neighboring…
FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference
The paper introduces FreqDepthKV, an inference-time cache compression method for long-context LLM (Large Language Model) inference that reduces memory and bandwidth costs while maintaining task accuracy on various…
