arXiv cs.CL· Qiutong Tony Yi, Wang Bill Zhu, Robin Jia, Jesse Thomason·· 12 小时前AI 评分52
研究称 ToM 基准主要考察状态跟踪而非心智理论
Algorithmic Primitives of Theory of Mind Benchmarks
AI 导读
论文提出 PDDL-Mind 神经符号方法，将叙事解析为 PDDL 并对候选动作执行预定义更新算法，从而向模型提供程序验证的状态轨迹。在 MMToM、MuMA 和 FanToM 上，仅把解析交给程序就能让性能接近强智能体流水线，把状态跟踪交给 PDDL 则让 GPT-4o 接近人类水平。
正文 · 原文
Abstract:Do Theory of Mind (ToM) benchmarks really test theory of mind, or are they actually testing parsing and algorithmic state tracking? While benchmarks vary modality and dialogue length, this paper finds that they re-encode the same few state transition algorithms. We create PDDL-Mind, a neuro-symbolic approach that recovers such algorithms and upstream inductive biases of ToM benchmarks explicitly by parsing narratives into the Planning Domain Definition Language (PDDL) and executes candidate actions against a predefined set of update algorithms. Doing so provides program-verified state traces to the model. We use our method's performance gain to audit how much of a benchmark's difficulty our inductive bias absorbs. Across MMToM, MuMA, and FanToM, outsourcing parsing to programs alone brings performance close to strong agentic pipelines, while outsourcing state tracking to PDDL moves GPT-4o to close-to-human performance. We demonstrate that complex ToM benchmarks primarily punishes Large Language Models' unreliable implicit implementation of state tracking and parsing algorithms rather than limitations in belief and counterfactual reasoning.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.17819 [cs.CL] |
| (or arXiv:2604.17819v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.17819 arXiv-issued DOI via DataCite |
Submission history
From: Qiutong Yi [view email]
[v1]
Mon, 20 Apr 2026 05:17:57 UTC (2,784 KB)
[v2]
Thu, 15 Oct 2026 10:22:39 UTC (2,950 KB)
来源：arXiv cs.CL · arxiv.org