arXiv cs.CL· Taiga Ikeda, Daisuke Miyashita, Jun Deguchi·· 6 天前AI 评分38
KV-Repurpose：评估非前缀 KV-Cache 复用的测试平台
KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse
AI 导读
基于 vLLM 的开源系统 KV-Repurpose 用于开发和评估超越精确前缀的分段式 KV-cache 复用。在 Zipfian RAG 式负载（20% 请求共享固定前缀）下，30 GB 缓存容量时其 token 级缓存命中率达 63.9%，而前缀缓存仅 3.3%，TTFT 更低；即便缓存容量扩至 1 TB，前缀缓存命中率也只有 17.9%。
正文 · 原文
Abstract:KV-cache reuse in large language model serving is commonly limited to exact prompt prefixes, preventing reuse when the same retrieved document appears after different system prompts, conversation histories, or dynamic metadata. We present KV-Repurpose, an open-source, vLLM-based system for developing and evaluating segment-wise KV-cache reuse beyond exact prefixes. The system combines cache-aware segment scheduling, prompt construction, capacity management, workload generation, and runtime monitoring. Experiments on long-context benchmarks show that naive segment reuse largely preserves output quality for single-document question answering and summarization, while exposing clear limitations in cross-document reasoning. Under a Zipfian RAG-style workload in which 20% of requests share a fixed prefix, KV-Repurpose achieves a substantially higher token-level cache hit rate than prefix caching at the same 30 GB cache capacity, 63.9% versus 3.3%, resulting in lower TTFT. Even with 1 TB of cache capacity, prefix caching reaches only a 17.9% hit rate in this workload. Source code is available under the Apache-2.0 license at this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13060 [cs.CL] |
| (or arXiv:2610.13060v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13060 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Taiga Ikeda [view email]
[v1]
Fri, 9 Oct 2026 06:17:10 UTC (844 KB)
来源：arXiv cs.CL · arxiv.org