FrontierPhysics：评估 AI 智能体端到端物理研究能力的基准
FrontierPhysics: Evaluating Agents for End-to-End Physics Research
研究团队推出 FrontierPhysics，一个由 42 位物理学家基于自身研究设计的 56 项端到端任务基准，覆盖理论、计算与实验物理九大领域，要求智能体完成文献调研、研究规划、实验与论文撰写。在 22 个模型、26 种配置中，没有任何配置在 168 次 rollout 中通过超过 46 次。人工审查显示，前沿智能体的主要短板是推理不够严谨和专家直觉有限，而非物理知识或执行错误。
Authors:Bingran You, Wenjun Ke, Ria Rosenauer, Bo Liu, Wenbo Chen, Min Chen, Shuqi Xu, Yue Ma, Jiechao Feng, Bo Chen, Xiangkai Sun, Jiarui Liu, Jinen Guo, Zeyu Zhou, Xu Huang, Zhuohan Li, Haoyu Li, Heguang Lin, Lijie Ding, Yadong Zeng, Kangqi Zhang, Xinge Liu, Hanwen Liu, Zhenghong Li, Alexander von Recum, Yuchen Zhu, Felix Schwarzfischer, Yizhou Zhao, Jicheng Wang, Binxu Li, Qian Yu, Chan Li, Cheng Dai, Di Wang, Haoyu Zhang, Jiankai Sun, Kaituo Feng, Mian Qin, Ruihong Yin, Yiqun Luo, Weiwei Wu, Shihong Pan, Tsung Yeh Hsieh, Xiao Huang, Yan Zhang, Sihan Li, Zirui Ren, Zengji Tu, Kyoung Whan Choe, Ke Sun, Vincenzo Galia, Vaisakhi Mishra, Jiashu He, Hongyang Wang, Zhiheng Yang, Shubham Gaur, Yifan Gu, Shuyi Wang, Zihao Zuo, Steven Dillmann, Yiyou Sun, Xinyang Han, Hanchung Lee, Molei Tao, Yue Zhao, Shilong Liu, Yuchen Wu, Xuandong Zhao, Xiangyi Li, Florian Brand, Gerbrand Ceder, Hartmut Haeffner
Abstract:AI agents are playing an increasingly important role in physics research. However, existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
| Comments: | 41 pages (10 pages main text), 19 figures, 11 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.06910 [cs.AI] |
| (or arXiv:2611.06910v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06910 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bingran You [view email]
[v1]
Wed, 4 Nov 2026 18:56:22 UTC (2,848 KB)
来源：arXiv cs.AI · arxiv.org