TheoPhysEval：评估 AI 智能体自主解决理论物理问题的能力
TheoPhysEval: Assessing AI agents at autonomous theoretical physics problem solving
研究团队发布 TheoPhysEval，包含 50 道未发表、答案可验证的研究级理论物理问题，用于测试 AI 智能体在无人类提示下自主解题的能力。在可检索文献并编写数值与符号脚本的条件下，表现最好的 GPT-6-Astra（运行于 Codex）准确率达 98%，平均每题成本低于 3 美元。禁用网络搜索和背景论文后，各智能体准确率仅有小幅变化，作者认为这提示模型对文献存在显著记忆。
Authors:A. Ramesh Chandra, Aaghaz Mahajan, Ainesh Sanyal, Mianqi Wang, Shubhadeep Chakraborty, Sudip Chakraborty, Shailesh Chandrasekharan, Rama Govindarajan, Akshit Goyal, Jonathan J. Heckman, Yasir Iqbal, Divya Jaganathan, Saumav Kapoor, Andreas Karch, Pushkar Khandare, Sandeep Krishna, Anupam Kundu, Alok Laddha, Tongyan Lin, Swarnendu Maity, Nilmani Mathur, Shani Nadir Meynet, Alessandro Mininno, Abhijeet Mishra, Vidur Niranjan, Abhishodh Prakash, Pratik Rath, Durbar Roy, Debargha Sarkar, Srikanth Sastry, Gary Shiu, Aninda Sinha, Thomas Van Riet, Praneeth Netrapalli, Suvrat Raju
Abstract:We present a suite of 50 unpublished research-level theoretical physics problems with verifiable answers. Without AI assistance, a typical problem in our collection could be solved by a world expert in about a day or less but would take considerably longer for experts in adjacent specialties. We test the ability of off-the-shelf AI agents to tackle these problems autonomously, without human hints. When given access to tools available in a realistic setting, including the ability to search the literature and write scripts for numerical and symbolic work, the best-performing agent, GPT-6-Astra harnessed in Codex, obtains the final answers with a near-perfect accuracy of 98% at an average cost under USD 3 per problem. On average, agents show only small changes in accuracy when denied web search and access to background research papers, suggesting significant memorization of the literature. These results suggest that state-of-the-art agents can accurately perform concrete calculations required across theoretical physics research. We comment on the broader implications of these findings and discuss possible explanations for the current lack of agent-led breakthroughs in theoretical physics despite their outstanding problem-solving abilities.
| Comments: | 8+48 pages (v2) refs added |
| Subjects: | High Energy Physics - Theory (hep-th); Statistical Mechanics (cond-mat.stat-mech); Strongly Correlated Electrons (cond-mat.str-el); Artificial Intelligence (cs.AI); General Relativity and Quantum Cosmology (gr-qc) |
| Cite as: | arXiv:2610.16733 [hep-th] |
| (or arXiv:2610.16733v2 [hep-th] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16733 arXiv-issued DOI via DataCite |
Submission history
From: Suvrat Raju [view email]
[v1]
Fri, 9 Oct 2026 13:47:49 UTC (451 KB)
[v2]
Thu, 12 Nov 2026 09:30:15 UTC (453 KB)
来源：arXiv cs.AI · arxiv.org