用强化学习让模型学会"感知到行动"规则，提升心智理论推理

Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning

AI 导读

研究用 GRPO 训练 Qwen2.5 和 Llama3，让模型生成连接感知与行动的 if-then 规则，在四个心智理论（ToM）基准上优于提示、监督微调和 think 推理。3B 以上大模型生成的规则还能迁移，提升 GPT-4o-mini 的下游表现，说明显式规则可弥合 ToM 推理差距并支持跨模型知识迁移。

正文 · 原文

View PDF HTML (experimental)

Abstract:In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting or judging the agent's subsequent actions. In this work, we investigate whether explicitly connecting perception, belief, and action with rules improves ToM reasoning. We generate concise \textit{if--then} rules that express relations between perception to action and use Group Relative Policy Optimization (GRPO) to train Qwen2.5 and Llama3 models across four ToM benchmarks. GRPO with rule generation outperforms prompting, supervised fine-tuning, and think reasoning. Moreover, rules generated by larger models ($\geq$3B) provide transferable knowledge that improves downstream performance of GPT-4o-mini. These findings suggest that explicit rules help models bridge the reasoning gap in ToM and support knowledge transfer across models.

Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as:	arXiv:2610.19796 [cs.AI]
	(or arXiv:2610.19796v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2610.19796 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Soyoung Oh [view email]
[v1] Wed, 14 Oct 2026 08:40:06 UTC (884 KB)

来源：arXiv cs.CL · arxiv.org