arXiv cs.CL· Eliza Berman, Rohan Subramanian, Shreya Jha, Lohit Jagarapu, Romain Lopez, Rajesh Ranganath·· 9 小时前AI 评分31
Solver2System:评估 LLM 在科学仿真中跨表示与执行推理的 PDE 求解器数据集
Solver to System: LLM Reasoning Across Representations and Execution in Scientific Simulation
AI 导读
研究者提出 Solver2System,一个包含多表示、受控损坏与词法扰动的 PDE 求解器数据集,用于评估 LLM 能否通过跨表示一致性和静态/智能体检测识别物理无效性。在八个开源权重模型上,LLM 在证据冲突时系统性偏好某些表示,对代码的词法扰动会改变其对其他未变表示的解释。执行能提升物理有效性判断,但较弱智能体可在未识别因果缺陷的情况下给出正确判断。
正文 · 原文
Abstract:Simulators based on partial differential equations (PDEs) are widely used to model physical systems, but generating and interpreting PDE solver code with large language models (LLMs) remains challenging. Scientific simulation code is a useful testbed for LLM scientific reasoning because (1) one system can be represented by code, equations, natural language, and numerical trajectories and (2) code can execute successfully while implementing invalid physics. We introduce Solver2System, a dataset of PDE solvers with multiple representations, controlled corruptions, and lexical perturbations, to evaluate whether LLMs can recover the underlying physical system through (1) cross-representation consistency and (2) static and agentic detection of physical invalidity. Across eight open-weight models, LLMs systematically favor some representations when evidence conflicts, and lexical perturbations to code alter interpretation of other unchanged representations. Execution improves physical-validity judgments, but weaker agents can judge correctly without identifying the causal defect. Thus, LLMs may reach correct judgments despite inconsistent representation reconciliation or causal misdiagnosis.
| Comments: | NeurIPS 2026 Workshop: Sim2Science: ML with Imperfect Scientific Models. 23 pages, 14 figures, 9 tables |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.21964 [cs.LG] |
| (or arXiv:2611.21964v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21964 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Eliza Berman [view email]
[v1]
Mon, 16 Nov 2026 22:47:37 UTC (5,060 KB)
来源:arXiv cs.CL · arxiv.org