arXiv cs.CL· Jack Shi, Jerry Gu·· 12 小时前AI 评分31
ContourKV：在物理强制预算下实现更优的 KV-Cache 淘汰规则
Trust the Mass: Forced Weights in KV-Cache Eviction
AI 导读
研究者提出免训练的 ContourKV，在物理强制预算下应用 KVzip 的评分规则，在 RULER 和 LongBench 的 160 次基准测试中有 93 次优于 KVzip，内存占用减少 28 至 1078 倍，并在同等内存下与最强基线 Compactor 持平。
正文 · 原文
Abstract:Every deployed KV-cache-eviction rule and every top-$k$ sparse-attention rule keeps a subset of the keys, discards the rest, and renormalizes the attention weights over the kept set. Identifying the exact best subset under that constraint, we find that keeping the largest weights is already near-optimal, since the best subset we find closes only a median $2$ to $5\%$ of the remaining gap to full attention. We evaluate deployed eviction methods in KVPress and find that methods with non-uniform per-head budgets apply attention masks over the full cache because dense cache tensors require equal length across heads. Truncating each head to the target budget lowers benchmark scores by $14$ to $62$ points, and computing token importance with the question in context accounts for $87.6$ points in retrieval accuracy. We propose ContourKV, a training-free rule that applies KVzip's score under a physically enforced budget. It beats KVzip in $93$ of $160$ benchmark runs across RULER and LongBench while using $28$ to $1078$ times less memory, and ties Compactor, the strongest baseline at equal memory.
| Comments: | Accepted at the Workshop on Foundations of Efficient Deep Learning, NeurIPS 2026. 22 pages, 5 figures |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.25230 [cs.LG] |
| (or arXiv:2608.25230v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2608.25230 arXiv-issued DOI via DataCite |
Submission history
From: Jerry Gu [view email]
[v1]
Tue, 25 Aug 2026 23:40:39 UTC (1,079 KB)
[v2]
Fri, 28 Aug 2026 06:26:03 UTC (1,079 KB)
[v3]
Thu, 5 Nov 2026 07:13:49 UTC (1,164 KB)
来源：arXiv cs.CL · arxiv.org