arXiv cs.AI· Chunkit Chan, Kawai Chung, Sze Heng Douglas Kwok, Yuxuan Liu, Yauwai Yim, Tianshi Zheng, Haochen Shi, Qing Zong, Wong Kai Chung, Pak Hin Lau, Calvin Yang, Yifan Gao, Xiaoman Pan, Ginny Wong, Simon See, Yangqiu Song·· 1 天前AI 评分26
MuSeToM：评估 LLM 能否依据六类感官文本线索判断信念修正
MuSeToM: Multi-Sensory Evaluation of Belief Revision with Textual Cues Across Six Senses
AI 导读
研究者提出 MuSeToM，一个经人工验证的纯文本基准，用于评估 LLM 在心理理论推理中能否判断感知证据是否足以支撑他人修正信念。该基准含 2,415 个改编自 BigToM 和 OpenToM 的实例，文本线索覆盖六类感官。对 SOTA 模型的评测显示明显割裂：模型常能察觉信念可能变化，却难以给出更新后的信念，并频繁在感知显著但证据不足的线索上过度修正。
正文 · 原文
Authors:Chunkit Chan, Kawai Chung, Sze Heng Douglas Kwok, Yuxuan Liu, Yauwai Yim, Tianshi Zheng, Haochen Shi, Qing Zong, Wong Kai Chung, Pak Hin Lau, Calvin Yang, Yifan Gao, Xiaoman Pan, Ginny Wong, Simon See, Yangqiu Song
Abstract:Theory of Mind (ToM), the ability to infer mental states in others, is pivotal for human social cognition. Existing evaluations of ToM in large language models (LLMs) mainly focus on static belief attribution and false-belief reasoning, leaving a critical aspect of social reasoning underexplored: can LLMs decide whether perceptual evidence is sufficient to justify another person's belief revision? To address this gap, we present MuSeToM, a human-validated, text-based benchmark for evaluating epistemic belief revision in ToM reasoning. MuSeToM contains 2,415 instances adapted from BigToM and OpenToM, with textual cues spanning six sensory categories, and tests whether models can determine when a protagonist should revise a belief and infer the resulting updated belief state. Using MuSeToM, we systematically evaluate state-of-the-art LLMs under direct and chain-of-thought prompting, revealing a pronounced dissonance: while models often detect that a belief may change, they struggle to identify the updated belief and frequently over-update on perceptually salient but epistemically insufficient cues.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.16662 [cs.CL] |
| (or arXiv:2611.16662v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.16662 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chunkit Chan [view email]
[v1]
Thu, 12 Nov 2026 11:38:52 UTC (8,533 KB)
来源：arXiv cs.AI · arxiv.org