arXiv cs.CL· Lucas Resck, Jingyi Sun, Qianli Wang, Isabelle Augenstein, Anna Korhonen·· 12 小时前AI 评分39
研究提出“依赖假说”：语言资源越少，CoT 忠实度越高
The Reliance Hypothesis: Chain-of-Thought Faithfulness Rises as Language Resource Level Falls
AI 导读
一项 arXiv 研究在八种语言的 MMMLU 平行选择题上发现，知识密集型任务中语言资源水平越低，CoT 忠实度越高，推理密集型任务则不然，该差距在四个模型和多种忠实度度量下方向一致。作者据此提出“依赖假说”：模型无法回忆答案时更依赖推理，模型置信度可部分解释资源水平效应，但无法解释知识与推理任务的差距。低资源语言中模型是对更多答案依赖推理，而非对每个答案依赖更强。
正文 · 原文
Abstract:Chain-of-Thought (CoT) faithfulness matters for monitoring, explaining and safely deploying large language models, yet it has been studied mostly in English. Languages differ in level, the amount of text available to learn from, so models know less in lower-resource languages, which may change how much they rely on their reasoning. To our knowledge, we are the first to relate CoT faithfulness to both resource level and task type (knowledge- or reasoning-intensive). On parallel multiple-choice questions from Multilingual Massive Multitask Language Understanding (MMMLU) in eight languages, faithfulness rises as resource level falls on knowledge-intensive tasks but not on reasoning-intensive ones, a gap that points the same way across four models and several faithfulness measures. We propose, and find support for, the Reliance Hypothesis: models rely more on their reasoning when they cannot recall the answer, as in lower-resource languages on knowledge-intensive tasks. Model confidence partly explains the effect of resource level, but not the knowledge-reasoning gap. Languages in which models are less confident rely more on reasoning, yet less confident answers within a language do not, a Simpson's paradox. In lower-resource languages, models rely on their reasoning for more answers, not more strongly for each. CoT monitoring may therefore be most informative in lower-resource languages, where it is least often validated.
| Comments: | 21 pages, 10 figures, 16 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.22841 [cs.CL] |
| (or arXiv:2610.22841v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22841 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jingyi Sun [view email]
[v1]
Fri, 16 Oct 2026 02:33:51 UTC (373 KB)
来源：arXiv cs.CL · arxiv.org