arXiv cs.CL· Mingyuan Li, Guangsheng Yu, Juyuan Zhang, Xu Wang, Zhibo Man, Haonan Zhang, Shaoxiong Ji·· 3 天前AI 评分22
MemoryAthena：在潜在记忆与生成记忆之间自适应路由
MemoryAthena: Adaptive Routing over Latent and Generated Memories
AI 导读
MemoryAthena 通过直接 Engram 检索（E）、基于检索线索的生成（GE）和不查表的因果骨干生成（GH）三条路径，用轻量因果路由头判断生成记忆何时介入。在冻结骨干、记忆、生成器与读取器的情况下，问答五任务平均分从 37.65 提升至 39.28，六任务通用 NLP 平均分从 76.73 提升至 79.13。记忆侧系统约含 201M 参数，不含冻结骨干。
正文 · 原文
Abstract:Learned-memory methods store information in an explicit table and consume it through a separate reader, allowing addressing, storage, and reading to be modified independently. We study whether useful memory can also be generated rather than only retrieved. MemoryAthena uses three pathways: direct Engram retrieval (E), generation from retrieved Engram cues (GE), and generation from causal backbone states without consulting the memory table (GH). Generated memory is conditionally useful: it can complement E in one context but interfere with it in another. MemoryAthena therefore treats E as an anchor and learns when a generated representation should intervene. With the backbone, memory, generators, and readers frozen, a lightweight causal routing head is trained from counterfactual future-token likelihood advantages of GE and GH relative to E. At inference time, an admitted candidate modifies the E residual through bounded interpolation, while rejection recovers the direct pathway exactly. On question answering, MemoryAthena raises the five-task average from 37.65 to 39.28 over the direct pathway of the same checkpoint, while the six-task general-NLP average increases from 76.73 to 79.13. The complete memory-side system contains approximately 201M parameters, excluding the frozen backbone. Further analyses show complementary strengths among E, GE, and GH across tasks and inputs. These results support generated memory as a selective correction to direct retrieval and highlight routing when, which, and how strongly to intervene as the central challenge.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.25853 [cs.CL] |
| (or arXiv:2609.25853v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.25853 arXiv-issued DOI via DataCite |
Submission history
From: Mingyuan Li [view email]
[v1]
Tue, 22 Sep 2026 08:17:43 UTC (2,266 KB)
[v2]
Sat, 26 Sep 2026 19:52:03 UTC (2,395 KB)
[v3]
Wed, 14 Oct 2026 08:23:38 UTC (2,397 KB)
来源：arXiv cs.CL · arxiv.org