arXiv cs.CL· Wanyang Hong, Zhaoning Zhang·· 2 天前AI 评分27
RoPERelay：稀疏 KV 缓存选择后的位置几何优化
Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
AI 导读
针对稀疏 KV 缓存中保留 token 仍沿用完整历史坐标、导致 RoPE 下被淘汰 token 的位置空隙持续干扰 query-key 匹配的问题，研究者提出 RoPERelay，为缓存键分配紧凑且保序的 RoPE 坐标，通过逆旋转与再旋转重定位键相位，并同步当前 query 与后续生成，无需重新选择、历史重预填充或额外缓存预算。
正文 · 原文
Abstract:Sparse KV compression reduces long-context inference costs by retaining selected historical states, yet these states typically keep their coordinates from the full history. Under Rotary Position Embedding (RoPE), gaps left by evicted tokens continue to affect query-key matching, potentially limiting the use of retained evidence. We study post-selection positional geometry on a fixed selector-produced cache, examining how positional organization affects the use of retained information and which coordinate and continuation rules improve answer quality. To reorganize this geometry, we propose RoPERelay, which assigns compact, order-preserving RoPE coordinates, relocates cached key phases through inverse rotation and re-rotation, and synchronizes the current query and subsequent generation. It reuses existing K/V without renewed selection, historical re-prefill, or additional cache budget. Across 36 comparisons spanning two models, six sparse selectors, and three long-term memory tasks, RoPERelay improves answer quality in 32 cases. For Llama-3.1-8B-Instruct with BM25 on LongMemEval-S, accuracy increases from 23.6% to 35.4%. Measured complete-request time overhead is approximately 1%-3% in the evaluated configurations. These findings establish positional organization as a distinct optimization dimension for sparse KV caches after content selection.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21126 [cs.CL] |
| (or arXiv:2610.21126v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21126 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wanyang Hong [view email]
[v1]
Thu, 15 Oct 2026 02:30:27 UTC (451 KB)
来源：arXiv cs.CL · arxiv.org