arXiv cs.AI· Shuxin Liu, Qing Liu, Yi Du, Ou Wu·· 3 天前AI 评分24
CORE：覆盖校准的 KV 保留与补偿，统一长上下文解码的保留与补偿双通道
One Distribution, Two Channels: Coverage-Calibrated KV Retention and Compensation
AI 导读
CORE 通过一个校准分布同时驱动 KV 保留与补偿两个通道，在 90% 压缩率下于三个骨干模型上最高超过最强 RULER 基线 3.78 分。该方法将查询效用与对数行列式覆盖的离线分配蒸馏为轻量缓存感知索引器，推理时无需单独的写入权重预测器或在线对数行列式评估。LongBench 与重复驱逐评测进一步验证了其有效性与解码效率。
正文 · 原文
Abstract:Long-context decoding is increasingly constrained by key--value (KV) cache memory and bandwidth. Existing fixed-budget compression methods typically separate retention from compensation, while a retention ranking specifies neither discarded attention mass nor the direction of induced output error. We start from an exact factorization: eviction error equals evicted attention mass times the directional gap between the evicted centroid and retained output, highlighting the importance of set-level coverage in retention and mass-preserving memory writing. We introduce CORE COverage Calibration and Evicted-Mass REdistribution for KV Cache, which distills an offline allocation combining query utility and log-determinant coverage into a lightweight cache-aware indexer. At inference, one calibrated distribution drives both channels: its Top-$B$ ordering retains complementary KV states, while its excluded allocation mass and conditional weights parameterize latent-memory writes without a separate write-weight predictor or online log-determinant evaluation. Our analysis provides a four-term pre-compensation error certificate, characterizes non-additive coverage interactions, and establishes mass-independent write stability with hierarchical bounds through recurrent and query-adaptive normalization. Across three backbones, CORE exceeds the strongest RULER baseline by up to 3.78 points at 90% compression; LongBench and repeated-eviction evaluations further demonstrate strong effectiveness and decoding efficiency.
| Comments: | Updated title metadata to match the PDF; manuscript content unchanged |
| Subjects: | Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.02235 [cs.AR] |
| (or arXiv:2610.02235v2 [cs.AR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02235 arXiv-issued DOI via DataCite |
Submission history
From: Shuxin Liu [view email]
[v1]
Mon, 28 Sep 2026 08:31:48 UTC (1,281 KB)
[v2]
Sat, 10 Oct 2026 07:52:25 UTC (1,284 KB)
来源：arXiv cs.AI · arxiv.org