arXiv cs.CL· Seojin Kim, Noseong Park·· 10 小时前AI 评分34
PRKV:基于全局重要性估计的 KV Cache 驱逐方法
Global Importance Estimation for KV Cache Eviction
AI 导读
PRKV 是一种免训练的 KV cache 驱逐方法,通过在注意力诱导的转移图上运行 Personalized PageRank 来估计多跳全局 token 重要性,无需修改模型即可让重要性传播超越近因或瞬时注意力。
正文 · 原文
Abstract:Memory consumption of the key-value (KV) cache remains a central bottleneck for long-context LLM inference. We introduce PRKV, a training-free KV eviction method that estimates multi-hop global token importance by applying Personalized PageRank over attention-induced transition graphs. This enables importance propagation beyond recency or instantaneous attention, without modifying the model. We benchmark PRKV on long-context understanding, sparse retrieval, and chain-of-thought reasoning tasks using multiple LLM families. Under moderate to aggressive compression ratios, including up to 90%, PRKV achieves competitive task performance against established eviction baselines while reducing peak memory usage with low end-to-end overhead. These results demonstrate that multi-hop global influence estimation yields a favorable accuracy--memory trade-off for scalable long-context inference.
| Comments: | The 40th Conference on Neural Information Processing Systems (NeurIPS 2026) |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24969 [cs.CL] |
| (or arXiv:2611.24969v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24969 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Seojin Kim [view email]
[v1]
Mon, 16 Nov 2026 02:29:03 UTC (5,004 KB)
来源:arXiv cs.CL · arxiv.org