arXiv cs.CL· Chen Chen, Rui Cheng·· 12 小时前AI 评分29
Write What Matters：将 token 选择与槽位分配解耦的循环记忆方法
Write What Matters: Disentangling Token Selection from Slot Allocation in Recurrent Memory
AI 导读
研究者提出 Factorized Write Routing（FWR），把循环记忆中的写入拆成两步：先决定每个 token 保留还是跳过，再选择固定或随 token 自适应的槽位数量，查询时仍读取全部槽位。
正文
Abstract:Long-sequence models must retain earlier information. Full attention keeps all past tokens but grows expensive, and recurrent-memory models compress history into fixed slots for linear-time processing. Limited capacity makes writes consequential: distractors can interfere with useful content, while skipped evidence cannot be recalled. Existing multi-slot memories write every token to the same number of slots, merging whether to store it with how many slots to update. We introduce Factorized Write Routing (FWR), which first keeps or skips each token and then chooses a fixed or token-adaptive number of slots. Queries still read all slots. Across ten seeds on two synthetic tasks that query facts after long contexts, we observe a density-dependent reversal. With four slots per write, long-context accuracy favors always writing on dense Feature Recall (0.472 vs 0.337), but learned selective writing on sparse-fact Passkey (0.222 vs 0.048). The tested adaptive rule underperforms fixed allocation under the learned gate on both tasks. A diagnostic oracle given fact labels unavailable at deployment reaches 0.372 on Passkey, showing headroom in filtering distractors. The results suggest avoiding missed writes in information-dense contexts and filtering distractors when they dominate.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.34180 [cs.CL] |
| (or arXiv:2610.34180v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34180 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chen Chen [view email]
[v1]
Sun, 25 Oct 2026 09:33:54 UTC (447 KB)
来源：arXiv cs.CL · arxiv.org