arXiv cs.AI· Cheng Zhang, Xuanning Cao, Zhicheng Zheng, Hui Xiong·· 3 天前AI 评分31
MemCut：LLM 智能体长期记忆中的每个秘密都会留下痕迹
MemCut: Every Secret Leaves a Trail in the Long-Term Memory of LLM Agents
AI 导读
研究者提出 MemCut，一个针对 LLM 智能体用户请求遗忘能力的系统性基准，包含 456 个手工设计的秘密及其完整标注的证据链，用于测试删除后的信息恢复。基准定义原生恢复与残差恢复两项指标，并在五种记忆设计和四种 LLM 上建立三个参考删除基线。结果显示，LLM 引导删除二十个检索候选中的额外十轮对话后，仍残留 39-76% 的恢复率，候选覆盖不足是现有基线的瓶颈。
正文 · 原文
Abstract:Long-term memory has become a crucial component of large language model (LLM) agents, with most research focusing on retrieving relevant information accurately and efficiently. However, a dependable memory system must also forget sensitive user data upon request. To evaluate this capability, we introduce MemCut, a systematic benchmark for user-requested forgetting. Its 456 hand-designed secrets in long-horizon conversations have fully annotated evidence trails to test recovery after deletion, including from messages that never state the secrets directly. We propose two recovery metrics. Native recovery measures reader success through the system's retrieval, whereas residual recovery measures the same reader's success when directly given surviving objects traced to annotated disclosures or evidence. We establish three reference deletion baselines to test how far existing retrievers and write models support forgetting across five memory designs and four LLMs. Disclosure removal reduces native recovery far more than residual recovery. LLM-guided deletion of ten additional turns from twenty retrieval candidates leaves 39-76% residual recovery. Subset sweeps (144 facts, three systems, two models) show that widening the pool at a fixed budget reduces residual recovery more than deleting all original candidates. These sweeps identify candidate coverage as a bottleneck in the reference baselines.
| Comments: | 17 pages, 4 figures, 13 tables, including appendices |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21405 [cs.AI] |
| (or arXiv:2610.21405v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21405 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhicheng Zheng [view email]
[v1]
Thu, 15 Oct 2026 07:23:03 UTC (1,988 KB)
来源：arXiv cs.AI · arxiv.org