FIELD NOTES TOPIC
llm inference
1 note, newest first.
FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference
The paper introduces FreqDepthKV, an inference-time cache compression method for long-context LLM (Large Language Model) inference that reduces memory and bandwidth costs while maintaining task accuracy on various…
