跳到正文
arXiv cs.AI· Ziv Kfir, Ido Kfir·· 4 小时前AI 评分34

KV-LTM:以持久化每会话 KV 缓存作为个人助理的长期记忆

KV-LTM: Persistent Per-Session KV Caches as Long-Term Memory for Personal Assistants

AI 导读

研究者提出 KV-LTM,将每个会话的原始文本与其 KV 缓存一同存储,回答时直接恢复缓存会话并追加问题,从而避免重新预填充。按 30K token/天场景测算,每用户年存储约 0.34-3.6 TB,而原始文本仅约 49 MB,单 token 状态约 31K-328K 字节。

正文 · 原文

View PDF HTML (experimental)

Abstract:Personal assistants accumulate months of conversations, but today's long-term memory stacks put derivation or selection between a question and its evidence. In a system-level comparison on LongMemEval, two commercial assistants scored well below the same model reading the history offline [1], consistent with, but not isolating, content loss between write and read. We propose KV-LTM: a memory tier storing each session's raw text plus its KV cache under a fixed reader model, answering by restoring cached sessions and appending the question. KV-LTM is complementary to, not a replacement for, retrieval and indexing. Text is the memory of record; the cache is a recompute-free prefill accelerator, degrading gracefully to text. We specify the design: storage record, budget-derived retention, append-only read path, and bounded aggregation. We size it analytically: across pinned BF16 full-attention (GQA) and latent-cached MLA configurations, per-token state spans approximately 31K-328K bytes (the pinned KDA+MLA hybrid differs: approximately 8.1K bytes/token plus a fixed per-session recurrent term), giving approximately 0.34-3.6 TB per user-year at a declared 30K-token/day scenario, against approximately 49 MB of raw text; per-query cost is modelled as regime-conditional transfer, attention, decode, aggregation, and compute. We define an evaluation protocol needing no persistent-cache implementation for logical QA evaluation: per-session prefill emulates valid cache reuse under the same model/input-state contract and ideal arithmetic, with named arms, controlled baselines, and a preregistered reader-model gate; a separate minimal real-cache replay tests numerical equivalence and measures the cost axis, with outcomes not assumed. KV-LTM trades storage for avoided re-prefill compute and write-time fidelity, within a declared retention budget.
Comments: 45 pages
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21998 [cs.AI]
  (or arXiv:2611.21998v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.21998

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ziv Kfir [view email]
[v1] Mon, 16 Nov 2026 23:30:00 UTC (41 KB)

来源:arXiv cs.AI · arxiv.org