arXiv cs.AI· Daniel Strau{\ss}, Zhenshan Bing, Alois Knoll·· 14 小时前AI 评分31
把检索变成遍历：面向有状态长上下文智能体的价值索引记忆
Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents
AI 导读
研究者提出“检索即遍历”框架，将记忆内容与地址分离，用成本正则化的索引目标让互补记忆可共同检索、冗余记忆不因单独有用而占据同一检索集。在 MiniGrid Memory 上仅测试最简单的固定几何实例：持久局部遍历头在五 token 上下文下用 0.59M 训练步达到 0.95 平均回报，而高斯绝对寻址需 0.94M、类别绝对寻址需超 2M。
正文 · 原文
Abstract:Long-context models repeatedly solve closely related retrieval problems. Attention or global retrieval can retain large stores, but each forward pass again searches for what matters. Recurrent compression avoids this search only by deciding earlier what information to preserve. Neither mechanism directly exploits that useful memories at neighbouring retrieval steps may substantially overlap. We propose an alternative in which retrieval is a persistent local traversal through an address space. We separate a memory's content from its address. Utility is defined by the information revealed by retrieving the content, while geometry determines only how cheaply that information can be reached. This leads to a cost-regularised indexing objective that preserves information value inside local neighbourhoods while charging for every memory made locally accessible. Its marginal-value form gives both attractive and exclusion pressures. Thus, complementary memories should be co-retrievable when the additional value exceeds retrieval cost, whereas redundant memories should not occupy the same retrieval set solely because they are individually useful. As a proof of principle, we test only the simplest fixed-geometry instance on MiniGrid Memory. A persistent local traversal head reaches $0.95$ mean return after $0.59$M training steps with a five-token context, compared with $0.94$M for Gaussian absolute addressing and more than $2$M for categorical absolute addressing. The experiment does not test learned geometry or repeated retrieval. Instead, it motivates the hypothesis that stateful retrieval may be easier to learn than stateless. To test the full proposed method further experimentation is necessary.
| Comments: | Accepted at the NeurIPS 2026 Workshop on Long-Context Foundation Models (LCFM). 7 pages, 1 table |
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.6; I.2.8 |
| Cite as: | arXiv:2610.40545 [cs.AI] |
| (or arXiv:2610.40545v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40545 arXiv-issued DOI via DataCite |
Submission history
From: Daniel Strauß [view email]
[v1]
Thu, 29 Oct 2026 17:37:21 UTC (14 KB)
来源：arXiv cs.AI · arxiv.org