arXiv cs.AI· Marta Kne\v{z}evi\'c, Seyed Parsa Neshaei, Jibril Frej, Vinitra Swamy, Tanja K\"aser·· 2 天前AI 评分26
如何评估 LLM 智能体的探究学习能力：假设、推理与探索
Evaluating Inquiry Learning in LLM-Based Agents: Hypothesizing, Reasoning, and Exploration
AI 导读
研究者提出一个统一交互式框架，用于评估 LLM 智能体的完整探究能力：将 Countdown 任务扩展为数学环境，让智能体通过操纵变量、收集观察来推断变量间关系。在 7 个 LLM 和多种提示策略下，实验显示模型固有探究能力有限、发现底层表达式的成功率偏低，但受教学脚手架启发的结构化引导能显著提升表现。
正文 · 原文
Abstract:Scientific inquiry involves iterative cycles of exploration, hypothesis generation, and reasoning over observations to uncover relationships in complex systems. Large language models (LLMs) have demonstrated strong capabilities in several components of this process, including inductive reasoning, hypothesis generation, and data analysis. However, prior work typically evaluates these abilities in isolation, leaving open how models integrate them across a full inquiry cycle. In this work, we introduce a framework for evaluating LLMs' inquiry capabilities in a unified, interactive setting. We extend the Countdown task to a mathematical environment where an agent probes the system by manipulating variables, collecting observations, and inferring relationships between them. This setup enables analysis of exploration strategies, causal discovery, and hypothesis refinement during inquiry. Across seven LLMs and multiple prompting strategies, our experiments reveal limited inherent inquiry capabilities and low success rates in discovering the underlying expressions. However, structured guidance inspired by educational scaffolding significantly improves performance. Our findings highlight current limitations of LLMs in strategic exploration and iterative hypothesis refinement, and show how simple guidance strategies developed for human learners in inquiry learning can support more effective inquiry behavior.
| Comments: | EMNLP 2026 (Findings) |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21604 [cs.AI] |
| (or arXiv:2610.21604v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21604 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Marta Knežević [view email]
[v1]
Thu, 15 Oct 2026 09:52:14 UTC (534 KB)
来源：arXiv cs.AI · arxiv.org