arXiv cs.AI· Chuangtao Chen, Grace Li Zhang, Xunzhao Yin, Cheng Zhuo, Bing Li, Ulf Schlichtmann·· 14 小时前AI 评分38
KV Packet：面向 LLM 的免重计算、上下文无关 KV 缓存
KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
AI 导读
KV Packet 是一种免重计算的 KV 缓存复用框架，将缓存文档视为不可变的“packet”，并用轻量可训练软 token 适配器包裹，通过自监督蒸馏弥合上下文不连续。在 Llama-3.1 和 Qwen-3 上的实验显示，该方法实现近零 FLOPs，TTFT 低于基于重计算的基线，同时 F1 分数与完全重计算基线相当。该工作已被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Large Language Models (LLMs) rely heavily on Key-Value (KV) caching to minimize inference latency. However, standard KV caches are context-dependent: reusing a cached document in a new context requires recomputing KV states to account for shifts in attention distribution. Existing solutions such as CacheBlend, EPIC, and SAM-KV mitigate this issue by selectively recomputing a subset of tokens; however, they still incur non-negligible computational overhead (FLOPs) and increased Time-to-First-Token (TTFT) latency. In this paper, we propose KV Packet, a recomputation-free cache reuse framework that treats cached documents as immutable "packets" wrapped in light-weight trainable soft-token adapters, which are trained via self-supervised distillation to bridge context discontinuities. Experiments on Llama-3.1 and Qwen-3 demonstrate that the proposed KV Packet method achieves near-zero FLOPs and lower TTFT than recomputation-based baselines, while retaining F1 scores comparable to those of the full recomputation baseline. Code and experimental results are available at this https URL.
| Comments: | Accepted at NeurIPS 2026. 18 pages, including references and appendices. Updated to the camera-ready content with additional experiments |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.13226 [cs.LG] |
| (or arXiv:2604.13226v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.13226 arXiv-issued DOI via DataCite |
Submission history
From: Chuangtao Chen [view email]
[v1]
Tue, 14 Apr 2026 18:50:47 UTC (779 KB)
[v2]
Fri, 17 Apr 2026 02:14:01 UTC (778 KB)
[v3]
Sun, 25 Oct 2026 13:55:56 UTC (966 KB)
来源：arXiv cs.AI · arxiv.org