arXiv cs.CL· Junjie Peng, You Wu, Haoyi Wu, Jialong Han, Xiaohua Xie, Kewei Tu, Jianhuang Lai·· 12 小时前AI 评分31
GRKV：面向长上下文 LLM 的免训练 KV Cache 压缩全局回归方法
GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs
AI 导读
GRKV 是一种免训练的 KV Cache 合并方法，通过岭回归将淘汰 token 的信息分摊到保留 token 上，直接最小化压缩缓存与完整缓存注意力输出的差异。在 LongBench 和 RULER 长上下文基准上，GRKV 是唯一能在开销极小的前提下提升整体性能的合并方法。该工作已被 EMNLP 2026 主会接收，代码已开源。
正文 · 原文
Abstract:Large language models (LLMs) with extended context lengths rely on the key-value (KV) cache to support attention over prior tokens. However, maintaining the KV cache incurs substantial memory overhead, motivating KV-cache compression methods that enforce a fixed budget through eviction and merging. Modern eviction methods increasingly adopt span-based retention because preserving contiguous spans is empirically effective and better preserves semantic coherence. Yet, when combined with post-eviction merging, span-based retention concentrates merges onto a small set of span-boundary carrier tokens, producing a highly imbalanced merge pattern that exacerbates over-merging and increases information loss. To address this imbalance, we propose GRKV (Global Regression for KV Cache), a training-free KV-cache merging method that directly minimizes the discrepancy between compressed-cache and full-cache attention outputs. GRKV uses ridge-regression-based merge steps to distribute information from evicted tokens across retained tokens, while regularizing the updates to prevent over-smoothing. Across the LongBench and RULER long-context benchmarks, GRKV is the only merging method that improves overall performance with minimal overhead. Our code is available at this https URL.
| Comments: | Accepted by EMNLP 2026 Main |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.31105 [cs.CL] |
| (or arXiv:2605.31105v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.31105 arXiv-issued DOI via DataCite |
Submission history
From: Junjie Peng [view email]
[v1]
Fri, 29 May 2026 10:16:30 UTC (2,004 KB)
[v2]
Mon, 31 Aug 2026 13:43:07 UTC (2,017 KB)
[v3]
Fri, 23 Oct 2026 05:39:09 UTC (2,016 KB)
来源：arXiv cs.CL · arxiv.org