arXiv cs.CL· Wei-Chieh Huang, Weizhi Zhang, Yuchen Wu, Yankai Chen, Eric Hanchen Jiang, Wooseong Yang, Yiwei Yang, Henry Peng Zou, Hanrong Zhang, Ying Nian Wu, Haolun Wu, Kai-Wei Chang, Philip S. Yu, Xue Liu, Aylin Caliskan·· 12 小时前AI 评分38
记忆基底如何选？LLM 智能体记忆基底的统一评测
Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents
AI 导读
一项被 NeurIPS 2026 接收的研究用 26 项性能与效率指标，在仅改变记忆基底的受控框架下统一评测了 11 种记忆方法，覆盖稠密与稀疏索引、文本记录、结构化存储、分层存储、参数化更新等，并用 3 个基座模型在 4 个基准套件上测试。
正文 · 原文
Authors:Wei-Chieh Huang, Weizhi Zhang, Yuchen Wu, Yankai Chen, Eric Hanchen Jiang, Wooseong Yang, Yiwei Yang, Henry Peng Zou, Hanrong Zhang, Ying Nian Wu, Haolun Wu, Kai-Wei Chang, Philip S. Yu, Xue Liu, Aylin Caliskan
Abstract:Memory is becoming core infrastructure for long-horizon LLM agents, yet existing evaluations offer limited guidance on which memory substrate, namely the underlying medium in which memory is represented and stored, should be used under different operating regimes. We introduce a unified suite of 26 performance and efficiency metrics, including auxiliary-LLM costs attributed to the write, read, and management stages, and apply it identically to 11 memory methods in a controlled harness that varies only the memory substrate. The methods cover dense and sparse indices, text records, structural stores, hierarchical stores, refinement-based memories, parametric updates, and activation-compatible context mechanisms, and we evaluate them with three backbone models on four benchmark suites spanning user-centric question answering and agent-centric decision-making. Our results show that no single substrate consistently dominates: broad retrieval benefits long-context factual QA, while excessive retrieval can harm sequential decision-making by shifting attention away from action-critical context. Scalability introduces a further routing axis, as substrates that perform well at moderate history lengths can become costly or brittle at longer horizons. These regime-dependent reversals make substrate selection a design axis. Choosing a substrate per observable operating regime improves over the best fixed substrate on our benchmarks, while query-level routing leaves most of the oracle headroom open. The findings provide empirical guidance for designing efficient, reliable, and regime-aware long-term memory for LLM agents.
| Comments: | Accepted at NeurIPS 2026, Evaluations and Datasets Track |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.15008 [cs.CL] |
| (or arXiv:2608.15008v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.15008 arXiv-issued DOI via DataCite |
Submission history
From: Wei-Chieh Huang [view email]
[v1]
Sat, 15 Aug 2026 03:29:48 UTC (2,271 KB)
[v2]
Wed, 4 Nov 2026 09:48:33 UTC (2,479 KB)
来源：arXiv cs.CL · arxiv.org