arXiv cs.CL· Yandan zheng, Haoran Luo, Xinliang Zhou, Anhtuan Luu·· 11 小时前AI 评分34
CogRSI：演化与审计认知推理策略
CogRSI: Evolving and Auditing Cognitive Reasoning Strategies
AI 导读
CogRSI 让语言模型通过自我改进循环自行写出推理原则，在 800 对未见过的 BigToM 测试上，学到的原则得分均高于专家撰写的原则。但将这些规则翻译成代码后在数千个已知答案的小故事上检验，多数规则在至少一种其声称覆盖的情境下给出错误答案，例如把熟睡中的在场角色判定为知道发生了什么。准确率无法揭示哪些规则有错，反事实测试也未能修正，策略在实践中有效与作为规则正确需要分别检验。
正文 · 原文
Abstract:Can a language model learn better ways to reason about other people's minds by improving its own strategies, and are the strategies it learns actually right? We study this with cogrsi. A grounded reader marks who saw each event in a story and uses a short written principle to settle disagreements between its readings. With a principle written by experts, it beats self-consistency at the same token cost. A self-improvement loop then writes the principle itself. It keeps an update only if accuracy rises by a calibrated margin, and in one variant only if it also passes counterfactual test pairs. The learned principles transfer: on 800 BigToM pairs that no system had seen, each of them scores higher than the expert principle. We then check what the learned rules actually say. Each rule is translated into code and run on thousands of small stories with known answers. Most of them give a wrong answer in at least one situation they claim to cover, for example by treating a character who is in the room as knowing what happened even while asleep. Accuracy does not reveal which rules are wrong, and the counterfactual test does not fix this. A strategy can help in practice and still be wrong as a rule, so both need checking. Our code and data are publicly available.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.01113 [cs.CL] |
| (or arXiv:2611.01113v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01113 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yandan Zheng [view email]
[v1]
Sat, 31 Oct 2026 16:59:29 UTC (1,029 KB)
来源：arXiv cs.CL · arxiv.org