arXiv cs.AI· Yijia Mao, Song Dai, Tao Chen, Xiangyu Lu, Linqian Zeng, Chenxi Hu, Sitong Zhou, Dongyang He, Xiaodan Mao, Linxuan Zhang, Tong Zhu, Yafu Li, Fangchen Yu, Peng Ye, Feng He, Yu Cheng, Yun Luo·· 2 天前AI 评分34
PhysClosureBench：用证据支撑的物理研究评估 AI 智能体的科研闭环能力
From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research
AI 导读
研究者提出 PhysClosureBench，包含 50 个经专家验证、目标明确的研究任务，覆盖强场、阿秒及邻近物理领域，用 20 项 Atomic Score 衡量局部科研成果，并以 Research Closure（RC0–RC4）评估研究问题的完整解决程度。
正文 · 原文
Authors:Yijia Mao, Song Dai, Tao Chen, Xiangyu Lu, Linqian Zeng, Chenxi Hu, Sitong Zhou, Dongyang He, Xiaodan Mao, Linxuan Zhang, Tong Zhu, Yafu Li, Fangchen Yu, Peng Ye, Feng He, Yu Cheng, Yun Luo
Abstract:Can AI agents organize scientific investigations that lead to well-supported conclusions? To evaluate this capability, we introduce PhysClosureBench, a benchmark of 50 expert-verified, goal-specified research tasks spanning strong-field, attosecond, and neighboring physics areas. Each task specifies the research objective and essential scientific constraints, while leaving the agent to organize the investigation and determine how to generate and validate the evidence needed to answer the question. A 20-criterion Atomic Score measures local scientific accomplishment, while Research Closure (RC0--RC4) assesses how fully the investigation resolves the specified research question based on whether its indispensable evidence requirements and applicable scope conditions are satisfied. We evaluate seven frontier LLMs, where Claude-Opus-5 achieves the highest mean Atomic Score (6.29/10) but reaches RC4 in only 32.7% of attempts. Across the benchmark, 36.2% of attempts scoring at least 8/10 still fall short of RC4, while all seven models reach RC4 on substantially fewer tasks in all three runs than in at least one. Case-level analysis further illustrates how misinterpretation of generated results and unresolved physical applicability limits can lead to incomplete closure. Together, these results reveal a persistent gap between AI agents' local scientific accomplishments and their ability to achieve complete research closure.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21767 [cs.AI] |
| (or arXiv:2610.21767v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21767 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yi-Jia Mao [view email]
[v1]
Thu, 15 Oct 2026 12:23:22 UTC (1,157 KB)
来源：arXiv cs.AI · arxiv.org