arXiv cs.CL· Hee Seung Hwang, Xindi Wu, Sanghyuk Chun, Olga Russakovsky, Zhiwei Deng·· 13 小时前AI 评分34
ReFINE：用下一序列预测强化快速权重架构的长上下文建模
Reinforced Fast Weights via Next-Sequence Prediction
AI 导读
研究者提出 ReFINE（Reinforced Fast weIghts via Next sEquence prediction），用序列级下一序列预测目标替代 NTP，通过预测熵筛选 token 位置、生成多 token rollout 并以 GRPO 优化。
正文 · 原文
Abstract:Fast weight architectures offer a promising alternative to standard transformers for long-context modeling by replacing the KV-cache with a recurrently updated fixed-size memory. Despite this architectural shift, they are typically trained with the same next-token prediction (NTP) objective as standard transformers. This creates a mismatch: fast weight models rely on an evolving memory state to support future predictions, while NTP provides only token-level supervision for the immediate next token. We address this mismatch with next-sequence prediction (NSP), a sequence-level extension of NTP that trains models to produce coherent multi-token continuations from their recurrent memory state. To optimize this sequence-level objective, we propose ReFINE (Reinforced Fast weIghts via Next sEquence prediction). ReFINE selects informative token positions based on prediction entropy, generates multi-token rollouts, assigns sequence-level rewards, and optimizes the model with Group Relative Policy Optimization (GRPO). ReFINE is applicable throughout the training lifecycle of pre-trained language models: mid-training, post-training, and test-time training. Experiments on LaCT-760M and DeltaNet-1.3B show that ReFINE consistently outperforms NTP-based supervised fine-tuning in needle-in-a-haystack retrieval, long-context question answering, and diverse tasks in LongBench. ReFINE provides an effective and versatile framework for improving long-context modeling in fast weight architectures.
| Comments: | NeurIPS 2026 (Spotlight) |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.16704 [cs.CL] |
| (or arXiv:2602.16704v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2602.16704 arXiv-issued DOI via DataCite |
Submission history
From: Hee Seung Hwang [view email]
[v1]
Wed, 18 Feb 2026 18:53:18 UTC (751 KB)
[v2]
Mon, 26 Oct 2026 18:53:20 UTC (782 KB)
来源：arXiv cs.CL · arxiv.org