arXiv cs.CL· Minyuan Ruan, Ziyue Wang, Kaiming Liu, Yunghwei Lai, Peng Li, Yang Liu·· 2 天前AI 评分31
BASIS：评估交互场景下超越指令的信念感知解决方案
BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions
AI 导读
研究者提出 BASIS 基准，用于评估 LLM 智能体能否推断用户请求背后的心理状态、识别信念与现实的不匹配，并据此给出合适方案。评估显示，当前模型在生成解决方案时往往忽略用户心理状态，而当这类信息被明确引出并提供时，方案质量持续更高。研究还通过通用 CoT 提示词对比、心理状态信息干预、针对性训练和多智能体协作实验，验证了考虑心理状态的实用价值。
正文 · 原文
Abstract:Large language model (LLM) agents are increasingly capable of carrying out complex, long-horizon tasks, yet effective assistance requires more than faithfully executing user instructions. Users' requests reflect their current understanding of the problem, which may be incomplete, outdated, or incorrect. When this understanding diverges from the actual task state, literal instruction following can fail to accomplish the user's underlying goal. We introduce \textbf{BASIS} (\emph{\textbf{B}elief-\textbf{A}ware \textbf{S}olutions in \textbf{I}nteractive \textbf{S}ettings}), a benchmark that evaluates whether agents can infer the mental states underlying user requests, recognize belief-reality mismatches, and use this understanding to develop appropriate solutions. Rather than assessing mental-state inference solely through explicit questions, \textbf{BASIS} evaluates both the utility of such inferences and the quality of downstream solutions in user-agent interactions. Evaluation results show that current models often fail to account for users' mental states when generating solutions, and that solution quality is consistently higher when such information is explicitly elicited and made available during problem solving. We further investigate the practical value of mental-state consideration through comparisons with generic Chain-of-Thought prompting, controlled interventions on mental-state information, targeted training, and multi-agent collaboration experiments under partial observability. Together, these studies highlight the importance of not merely inferring others' mental states, but also using those understandings to provide grounded, goal-directed assistance.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.13832 [cs.CL] |
| (or arXiv:2602.13832v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2602.13832 arXiv-issued DOI via DataCite |
Submission history
From: Minyuan Ruan [view email]
[v1]
Sat, 14 Feb 2026 16:01:59 UTC (1,743 KB)
[v2]
Thu, 15 Oct 2026 03:02:52 UTC (1,099 KB)
来源：arXiv cs.CL · arxiv.org