arXiv cs.CL· Lanni Bu, Xiulin Yang, Christian Clark, Alex Warstadt, Ethan Gotlieb Wilcox·· 12 小时前AI 评分31
超越静态与线性：哪种注意力约束最契合人类阅读时间？
Beyond Static and Linear: What Attention Constraints Best Fit Human Reading Times?
AI 导读
研究系统比较了多种基于注意力的记忆约束机制在不同模型规模和训练语料下对人类阅读时间的心理测量预测力与语法能力。结果显示，对中间 token 内容敏感的约束与人类阅读时间的对齐度最高，优于基于距离的约束。动态记忆课程下心理测量拟合与语法能力出现分离，表明 Transformer 难以充当通用的认知模型。
正文 · 原文
Abstract:Transformer-based language models are widely used as models of human language processing, yet their attention mechanisms allow lossless access to the full preceding context, unlike the limited memory systems of humans. We hypothesize that installing memory constraints into transformers' attention mechanisms can improve their fit to human behavioral data. While previous work has explored individual constraints in isolation, we conduct a systematic comparison of multiple attention-based memory mechanisms across different model sizes and training corpora, evaluating both psychometric predictive power for human reading times and grammatical competence. We additionally compare static constraints, in which the constraint strength is fixed throughout training, to dynamic memory curricula. We find that constraints that are sensitive to the content of intervening tokens consistently achieve the highest alignment with human reading times, outperforming distance-based constraints. We observe a dissociation between psychometric fit and grammatical competence under dynamic memory curricula, suggesting that Transformers cannot serve as a one-size-fits-all cognitive model.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.23818 [cs.CL] |
| (or arXiv:2608.23818v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.23818 arXiv-issued DOI via DataCite |
Submission history
From: Lanni Bu [view email]
[v1]
Mon, 24 Aug 2026 20:54:29 UTC (116 KB)
[v2]
Fri, 23 Oct 2026 17:08:52 UTC (109 KB)
来源：arXiv cs.CL · arxiv.org