arXiv cs.CL· Qituan Zhang, Guo Chen, Linfeng Zhang, Wenlong Zhang·· 12 小时前AI 评分34
PHYRD：用于复现与扩展理论物理研究的活基准
PHYRD: A Living Benchmark for Reproducing and Extending Theoretical Physics Research
AI 导读
PHYRD 是一个理论物理论证活基准，包含 63 道 Reproduce 题和 43 道 Discover 题，分别测试已发表结果的推导与修改理论条件后的推论。
正文 · 原文
Abstract:Large language models and agents have shown significant power in advanced physical research. In this paper, We introduce PHYRD, a living benchmark for theoretical-physics reasoning with 63 Reproduce and 43 Discover problems. Reproduce tests derivation of published results; Discover tests consequences of modified theoretical conditions. FormulaForest constructs the tasks with reference solutions and subquestion-level rubrics. Across five models, whole-problem accuracy ranges from 49.21% to 68.25% on Reproduce and from 18.60% to 74.42% on closed-book Discover. With tools, four models complete more Discover problems, but conceptual errors and missing arguments persist. Besides, our experimental results demonstrate the value of FormulaForest as synthetic data. On FrontierScience, a benchmark developed by OpenAI, it raises the accuracy of a Qwen-35B-A3B model from 61% to 72%.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.32575 [cs.CL] |
| (or arXiv:2610.32575v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32575 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qituan Zhang [view email]
[v1]
Fri, 23 Oct 2026 13:03:48 UTC (1,662 KB)
来源：arXiv cs.CL · arxiv.org