跳到正文
arXiv cs.CL· Seojin Kim, Noseong Park·· 10 小时前AI 评分34

PRKV:基于全局重要性估计的 KV Cache 驱逐方法

Global Importance Estimation for KV Cache Eviction

AI 导读

PRKV 是一种免训练的 KV cache 驱逐方法,通过在注意力诱导的转移图上运行 Personalized PageRank 来估计多跳全局 token 重要性,无需修改模型即可让重要性传播超越近因或瞬时注意力。

正文 · 原文

View PDF HTML (experimental)

Abstract:Memory consumption of the key-value (KV) cache remains a central bottleneck for long-context LLM inference. We introduce PRKV, a training-free KV eviction method that estimates multi-hop global token importance by applying Personalized PageRank over attention-induced transition graphs. This enables importance propagation beyond recency or instantaneous attention, without modifying the model. We benchmark PRKV on long-context understanding, sparse retrieval, and chain-of-thought reasoning tasks using multiple LLM families. Under moderate to aggressive compression ratios, including up to 90%, PRKV achieves competitive task performance against established eviction baselines while reducing peak memory usage with low end-to-end overhead. These results demonstrate that multi-hop global influence estimation yields a favorable accuracy--memory trade-off for scalable long-context inference.
Comments: The 40th Conference on Neural Information Processing Systems (NeurIPS 2026)
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24969 [cs.CL]
  (or arXiv:2611.24969v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.24969

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Seojin Kim [view email]
[v1] Mon, 16 Nov 2026 02:29:03 UTC (5,004 KB)

来源:arXiv cs.CL · arxiv.org