arXiv cs.CL· Jiacheng Cai, Tao Zhang, Siqi Liang, Yun Dai, Ziqian Zeng·· 11 小时前AI 评分34
超越 K/V 缓存：混合线性注意力 LLM 的跨模型状态交接
Beyond K/V Caches: Cross-Model State Handoff in Hybrid Linear-Attention LLMs
AI 导读
研究提出 Hybrid State Transfer（HST），在混合线性注意力 LLM 间同时交接 FA 缓存、循环状态（REC）与卷积历史（Conv）三类状态。在五个基准上，Qwen3.5 4B→27B 与 9B→27B 的平均任务保留率分别为 89.88% 和 94.11%；在源缓存已就绪时，合成 16K 输入上的交接主体加速达 21.66× 和 21.91×。
正文 · 原文
Abstract:Production deployments often use different models to balance cost and quality through cascading, routing, and switching models during long conversations. Context accumulates across turns. Each switch requires the target to repeat prefill over this context. Existing methods transfer full-attention (FA) K/V caches across models. Hybrid linear-attention models also retain recurrent states (REC) and convolution histories (Conv). The interaction between Conv and REC complicates cross-model state transfer. Conv perturbations can change keys and values in recurrent layers and alter REC updates. Conv histories are overwritten, but changes in REC can still affect later outputs. More accurate REC reconstruction need not improve continuation. Building on this, we propose Hybrid State Transfer (HST) for all three state types. It uses structured REC maps and preserves history order during Conv alignment. We calibrate the adapters using the frozen target's outputs during continuation. FA cache map updates are merged offline. Across five benchmarks, Qwen3.5 4B$\rightarrow$27B and 9B$\rightarrow$27B achieve mean task retentions of 89.88% and 94.11%, respectively. With source caches already available, separate timing on synthetic 16K inputs yields handoff body speedups of $21.66\times$ and $21.91\times$.
| Comments: | 12 pages, 4 figures, 9 tables |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.08133 [cs.CL] |
| (or arXiv:2611.08133v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08133 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiacheng Cai [view email]
[v1]
Thu, 5 Nov 2026 15:00:50 UTC (351 KB)
来源：arXiv cs.CL · arxiv.org