arXiv cs.CL· Zhenshuai Yin, Yongxin Guo, Mingda Zhang, Zhongze Luo, Haoyue Liu, Lianmin Zhang, Xiaoying Tang·· 15 小时前AI 评分38
从信念推理到任务行动：心智理论在可验证任务上的多层级评估
From Belief Inference to Task Action: A Multi-Level Evaluation of Theory of Mind on Verifiable Tasks
AI 导读
研究提出一套基于可验证任务的多层级评估，将心智理论（ToM）从故事问答延伸到实际决策与执行，用独立接口分别考察世界理解、信息获取、信念、决策和执行。
正文 · 原文
Abstract:Effective collaboration requires language models to understand what others know and believe, and to act accordingly. Models that assume others share their information may omit necessary communication and undermine coordination. Many theory-of-mind (ToM) benchmarks assess mental-state inference through story questions, but correct answers do not establish that models can use these judgments in subsequent tasks. We introduce a multi-level evaluation built on existing, verifiable tasks. We extend household collaboration and calendar scheduling with asymmetric information while retaining their original constraints and validators. Independent interfaces assess world understanding, information access, beliefs, decisions, and execution. Controls supply personal evidence or correct beliefs to distinguish state recovery from state use. Across six model configurations, these abilities diverge. On matched scheduling cases, Claude recovers all agent beliefs correctly but produces subjectively optimal plans in only 67.9% of cases. In household collaboration, supplying beliefs raises Qwen-27B success from 55.0% to 75.0%, yet Qwen-4B answers every fixed supplied-belief source question correctly while completing only 20.0% of the corresponding continuation condition. These results show why ToM evaluation must connect perspective inference to independently verified decisions and task consequences.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.34759 [cs.CL] |
| (or arXiv:2610.34759v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34759 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhenshuai Yin [view email]
[v1]
Mon, 26 Oct 2026 03:06:04 UTC (6,361 KB)
来源：arXiv cs.CL · arxiv.org