arXiv cs.CL· Ryan Cotterell·· 10 天前AI 评分24
Surprisal 理论是同义反复（缺乏理性基础）
Surprisal Theory is Tautological (without Rational Grounding)
AI 导读
一篇 EMNLP 2026 论文指出，Surprisal 理论主张语言单位的加工难度是其在大语言模型下 surprisal 的仿射函数，但该主张在缺乏额外约束时是同义反复：对任意非负难度度量，都存在一个语言模型使其 surprisal 与之成仿射关系，因此该理论无法做出可证伪预测。
正文 · 原文
Abstract:Surprisal theory holds that the human processing difficulty of a linguistic unit in context is an affine function of its surprisal under some language model. I argue this claim is a tautology without further constraint: for any non-negative difficulty measure over units in context, there exists a language model whose surprisal is an affine function of it under mild technical conditions. Therefore, because any pattern of difficulty is consistent with some language model, without an additional constraint on the language model, surprisal theory makes no falsifiable predictions. The tautology was long obscured by an assumption implicit in two decades of psycholinguistic work---that the relevant language model is the distribution that generated the training corpus, so that improving corpus fit improves predictions of human behavior. Recent empirical work has undermined this assumption, demonstrating that better corpus models can be worse predictors of processing difficulty. I conclude that breaking the tautology requires a rationalist intervention, i.e., the relevant language model must be derived from a non-empirically motivated model of the comprehender, which could be based on, for instance, memory constraints or processing goals, and that, thus, does not depend on the behavioral data surprisal theory is meant to explain.
| Comments: | EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.21574 [cs.CL] |
| (or arXiv:2607.21574v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.21574 arXiv-issued DOI via DataCite |
Submission history
From: Ryan Cotterell [view email]
[v1]
Thu, 23 Jul 2026 17:54:37 UTC (44 KB)
[v2]
Tue, 6 Oct 2026 19:31:53 UTC (46 KB)
来源：arXiv cs.CL · arxiv.org