arXiv cs.CL· Melanie Subbiah, Chia-En Jacklyn Tsai, Kathleen McKeown·· 12 小时前AI 评分38
ToMStories：用合成叙事评测语言模型的心理理论推理能力
ToMStories: Interpreting Theory-of-Mind in Synthetic Narratives
AI 导读
研究者构建了 ToMStories 评测数据集，包含约 6,000 个合成短叙事片段，每个片段配一道关于角色情绪、恐惧、动机或情感创伤的选择题或开放题，心理状态以隐含、明示或对话与描述相冲突三种方式呈现。
正文 · 原文
Abstract:Understanding people often requires inferring what they feel or want from how they behave rather than from what they state outright. We ask whether language models can make these inferences, and how they do so. We introduce Theory-of-Mind Stories (ToMStories), an evaluation dataset of ~6,000 short synthetically generated narrative vignettes, each paired with a multiple-choice or open-ended question about a character's emotion, fear, motivation, or emotional wound. These psychological states are conveyed implicitly (showing), explicitly (telling), or with conflict between dialogue and description. Evaluating thirteen open-weight models (1B-72B parameters), we find that the largest models match humans on multiple choice but not open-ended questions. Our analyses point to simplifications models make in both interpretation and generation of implicit psychological states, indicating that synthetic data can help us understand how models perform interpretation, but likely overestimates how well models interpret humans.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.22632 [cs.CL] |
| (or arXiv:2610.22632v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22632 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Melanie Subbiah [view email]
[v1]
Thu, 15 Oct 2026 20:38:58 UTC (10,758 KB)
来源：arXiv cs.CL · arxiv.org