arXiv cs.AI· Oleeviya Babu Poikarayil, Tony Basil Babu, Abdulrahman Nahhas, Het Darshan Mehta, Jawid Ahmad Baktash, Ernesto William De Luca, Klaus Turowski·· 11 小时前AI 评分38
研究揭示：Agent 记忆检索到正确事实仍不够，过时陈述会显著拉低准确率
When Retrieval Is Not Enough: Obsolete Facts in Agent Memory
AI 导读
在 MemoryAgentBench 的 FactConsolidation 任务上，Qwen2.5-7B-Instruct 的 BM25 检索证据预算从 64 tokens 增至 512 tokens 时准确率反从 45% 降至 32%，尽管每次检索集都含最新陈述。
正文 · 原文
Abstract:Memory-agent benchmarks often collapse retrieval and answer generation into one score, making it hard to tell whether a failure comes from missing an update or from not using an update that was already retrieved. We examine this distinction on the single-hop FactConsolidation task of MemoryAgentBench. With Qwen2.5-7B-Instruct on 6K-history BM25 accuracy drops from 45% to 32% when the retrieved evidence budget grows from 64 to 512 tokens, even though a gold-assisted post-hoc audit finds the newest statement in every retrieved set. Most of the errors occur on questions that also contain an obsolete statement version. Under a frozen prompt and equal four-statement evidence size, replacing one unrelated statement with the obsolete competing statement reduces accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B, with all 50 harmful paired flips classified as stale. A 32K-history replication likewise favors BM25-64 over BM25-512 (69% versus 58%). These results suggest that retrieving the correct fact is not enough. An obsolete competing statement can be more harmful than an unrelated piece of evidence of the same size.
| Comments: | Accepted at the NeurIPS 2026 Workshop on Personalized, Aligned, Long-Term Memory for AI Systems (PALM). 23 pages, including supplementary material |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36308 [cs.AI] |
| (or arXiv:2610.36308v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36308 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jawid Ahmad Baktash [view email]
[v1]
Mon, 26 Oct 2026 21:30:26 UTC (136 KB)
来源：arXiv cs.AI · arxiv.org