arXiv cs.AI· Yanqi Yu, Pingwei Sun, Jianchao Tan, Tao Zhang, Yuchen Xie, Xunliang Cai, Yao Liu·· 6 天前AI 评分22
DASC：面向混合线性注意力推理的衰减感知状态压缩
DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving
AI 导读
针对混合线性注意力模型前缀复用需存储循环状态检查点、加剧显存压力的问题，研究者提出衰减感知状态压缩 DASC，依据 Gated DeltaNet 与 Kimi Delta Attention 的衰减结构推导各头与通道的保留时域，筛选长时域状态单元并打包为不规则检查点布局，同时在 TP rank 间做均衡。
正文 · 原文
Abstract:Hybrid linear-attention architectures have recently scaled to large open-weight models, offering quality competitive with full attention while substantially reducing key/value (KV) cache growth. However, their in-place recurrent-state updates complicate cache management: prefix reuse requires state checkpoints alongside full-attention KV, while storing state checkpoints in full increases memory pressure, leading to more evictions and repeated prefill. By analyzing the decay structure of Gated DeltaNet (GDN) and Kimi Delta Attention (KDA), we find that different heads and channels retain prefix information over markedly different timescales, which we term \emph{retention horizons}. This variation suggests substantial compression potential in persistent state checkpoints. Building on this observation, we introduce \emph{Decay-Aware State Compression} (DASC), which derives retention horizons from model weights, selects long-horizon state units, and packs them into a ragged state checkpoint layout. To integrate efficiently with tensor-parallel inference engines, DASC furtherly balances compressed state checkpoints across TP ranks. On reuse, DASC either zero-fills omitted units or refreshes them from a bounded suffix with additional compute cost. Across retrieval and end-to-end reasoning benchmarks on Kimi-Linear, conservative DASC configurations remain close to full caching while compressing KDA recurrent state checkpoints by $2.63\times$. Under fixed state checkpoint memory budgets, the resulting capacity gains reduce mean Time to First Token (TTFT) by 42.6\% and improve input throughput by 68.4\%. At larger compression ratio, suffix refresh recovers much of the accuracy lost to more aggressive omission, at the cost of additional replay computation. Qwen with GDN exhibits a similar quality--efficiency trend, showing that DASC extends from channel-wise KDA to head-wise GDN.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.30386 [cs.LG] |
| (or arXiv:2608.30386v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2608.30386 arXiv-issued DOI via DataCite |
Submission history
From: Pingwei Sun [view email]
[v1]
Mon, 31 Aug 2026 07:42:43 UTC (467 KB)
[v2]
Fri, 9 Oct 2026 07:27:01 UTC (559 KB)
来源：arXiv cs.AI · arxiv.org