arXiv cs.CL· Jiwan Chung, Heechan Choi, Seon Joo Kim·· 13 小时前AI 评分35
重新审视循环模型中的状态追踪：误差控制动力学视角
Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
AI 导读
研究证明，仿射循环网络（涵盖 State-Space Models 与 Linear Attention）在保持状态表示后，无法沿状态区分子空间纠正误差，因此实际仿射追踪器学到的是受累积状态相关误差支配的有限时域解，而非鲁棒状态追踪。
正文 · 原文
Abstract:The theory of state tracking in recurrent architectures has predominantly focused on expressive capacity: whether a fixed architecture can theoretically realize a set of symbolic transition rules. We argue that equally important is error control, the dynamics governing hidden-state drift along the directions that distinguish symbolic states. We prove that affine recurrent networks, a class of models encompassing State-Space Models and Linear Attention, cannot correct errors along state-separating subspaces once they preserve state representations. Consequently, practical affine trackers do not learn robust state tracking; rather, they learn finite-horizon solutions governed by accumulated state-relevant error. We characterize the mechanics of this failure, showing that tracking remains readable only while the accumulating within-class spread remains small relative to the between-class separation. We demonstrate empirically on group state-tracking tasks that this breakdown is predictable: tracking collapses when the distinguishability ratio crosses the readability threshold. Across trained models, the point of this crossing predicts the horizon at which downstream accuracy fails. These results establish that robust state tracking is determined not only by an architecture's theoretical expressivity but crucially by its error control.
| Comments: | NeurIPS 2026. Project page: this https URL |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.07755 [cs.LG] |
| (or arXiv:2605.07755v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.07755 arXiv-issued DOI via DataCite |
|
| Journal reference: | NeurIPS 2026 |
Submission history
From: Jiwan Chung [view email]
[v1]
Fri, 8 May 2026 13:59:15 UTC (1,403 KB)
[v2]
Tue, 27 Oct 2026 04:03:20 UTC (1,418 KB)
来源：arXiv cs.CL · arxiv.org