arXiv cs.CL· Teresa Delgado, Hugo Hazard, Martin Benfeghoul, Jun Wang, Zafeirios Fountas·· 11 小时前AI 评分34
LowRAM：用低秩键与摊销值实现可组合的测试时 KV cache 压缩
LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time
AI 导读
LowRAM 提出一种非对称 KV cache 压缩流程，将跨层低秩键压缩与值缓存摊销结合，在三个模型家族上接近无损。其值摊销在测试时为每个注意力头拟合轻量预测器、仅存重建最差 token 的残差，实现 4× 近无损值压缩；4× 压缩下峰值解码吞吐最高提升 5×。LowRAM 还可与量化、驱逐组合，在长上下文下达到 26.9× 压缩并保留基线 94.5% 的性能。
正文 · 原文
Abstract:Autoregressive decoding relies on the KV cache, whose memory footprint grows linearly with context length and caps batch size and throughput. Existing post-training methods exploit redundancy across tokens, layers, and feature dimensions, or lower numerical precision. We identify a complementary axis: within each attention head, values are largely predictable from their corresponding keys. We exploit this with value cache amortisation, which fits a lightweight per-head predictor at test time and stores residuals only for the tokens it reconstructs worst, giving $4\times$ near-lossless value compression. The predictor can be initialised by meta-learning for substantially reduced prefill overhead, or fitted from scratch in settings that allow a one-off cost, such as multi-query inference. We pair value amortisation with cross-layer low-rank key compression to produce LowRAM, an asymmetric compression pipeline that stays close to lossless across three model families. By reconstructing only a selected subset of KV pairs at each decoding step, LowRAM can keep decoding overhead minimal and achieve up to $5\times$ higher peak decode throughput at $4\times$ compression. LowRAM further composes with quantisation and eviction, reaching $26.9\times$ compression at $94.5\%$ of baseline performance on long contexts. These results show that treating keys and values as structurally distinct enables efficient, aggressive, and composable KV cache compression.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.07025 [cs.LG] |
| (or arXiv:2611.07025v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07025 arXiv-issued DOI via DataCite |
Submission history
From: Teresa Delgado De Las Heras [view email]
[v1]
Wed, 4 Nov 2026 19:00:06 UTC (228 KB)
来源：arXiv cs.CL · arxiv.org