arXiv cs.AI· Kawai Chung, Chunkit Chan, Yuxuan Liu, Yauwai Yim, Xi Yang, Wenbin Hu, Huihao Jing, Chiu Yeung Tang, Changxuan Fan, Ki Sen Hung, Tsun On Kwok, Haoran Li·· 3 天前AI 评分24
Jev-ToM：在类型化决策模型中评估心智理论
Jev-ToM: Evaluating Theory of Mind in Typed Decision Models
AI 导读
Jev-ToM 在来自 9 个心智理论（ToM）基准、覆盖 6 类能力的 11,117 道英文文本问题上，对比了 5 个类型化决策模型与 7 个生成式基线。TypeSafe Jev 取得 73.2% 的 source-balanced 准确率（含原生控制项），在全部 6 类能力上排名第一或第二，并在信念、目标/欲望/意图和情感上领先。
正文 · 原文
Authors:Kawai Chung, Chunkit Chan, Yuxuan Liu, Yauwai Yim, Xi Yang, Wenbin Hu, Huihao Jing, Chiu Yeung Tang, Changxuan Fan, Ki Sen Hung, Tsun On Kwok, Haoran Li
Abstract:Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. Typed decision models offer a potential alternative to generative large language models for bounded Theory of Mind (ToM) selection. We present Jev-ToM, comparing five typed decision models and seven generative baselines on 11,117 English text questions from nine ToM benchmarks across six capability groups. TypeSafe Jev reaches 73.2% source-balanced accuracy including native controls and ranks first or second across all six capabilities, leading beliefs, goals/desires/intentions, and emotions. Other typed models show distinct strengths across tasks. Separate resource comparisons suggest that Jev could reduce the cost and waiting time of repeated ToM selection. Confidence-based selection remains benchmark-dependent, and supplied reasons can repair errors or overturn correct answers depending on their attachment. Candidate expansion reveals distinct selection and output-validity problems. Related-question cues account for most of Jev's downstream gains, with task-dependent effects from predicted states. The findings motivate testing Jev as a ToM selection component in assistants, with task-specific confidence calibration, candidate design, and output validation as integration priorities.
| Comments: | Code and evaluation materials: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18610 [cs.CL] |
| (or arXiv:2610.18610v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18610 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kawai Chung [view email]
[v1]
Tue, 13 Oct 2026 15:59:00 UTC (8,120 KB)
来源：arXiv cs.AI · arxiv.org