arXiv cs.AI· Shaoliang Yang, Jun Wang·· 14 小时前AI 评分52
语言模型在已知力学极限测试中计算与拒绝出现分歧
Calculation and rejection diverge in language models tested at known mechanical limits
AI 导读
研究者在梁载荷和盘转速设为已知塑性坍塌或爆裂阈值上下一定百分比的条件下，测试了12个模型共4,860次调用，发现模型被单独问及设计是否可行时能识别不可行设计，但被要求给出完整答案时却对同一设计报“solved”。
正文 · 原文
Abstract:When a language model returns engineering quantities with the status "solved", software that trusts it passes them on, even when the requested design is physically impossible. Such failures could reflect poor calculation, how the answer is requested, or a failure to judge feasibility. We set beam loads and disk speeds at known percentages above or below plastic collapse or burst, measured each model's limit calculation separately and tested 12 models in 4,860 calls, including follow-up experiments on new designs. We find that a model can recognize an impossible design when asked only whether it is feasible, yet accept the same design, reporting "solved", when asked for a full answer. Three models accepted beams 10 to 30% beyond collapse despite precise limit calculations in separate questions, and many accepted replies stated the violation. Asked only about feasibility, Opus 5 and Gemini 3.8 Flash rejected every impossible new design they were given, including those accepted in full answers, without false alarms. Rewording when to decline or changing the answer format raised rejection on the original beams, but not consistently on new designs. Software and benchmarks using a model's status should test feasibility judgment and full-answer rejection separately, and report the exact prompt.
| Comments: | 94 pages, including Supplementary Information. Data and code: this https URL |
| Subjects: | Computational Engineering, Finance, and Science (cs.CE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.38096 [cs.CE] |
| (or arXiv:2610.38096v1 [cs.CE] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38096 arXiv-issued DOI via DataCite |
Submission history
From: Shaoliang Yang [view email]
[v1]
Wed, 28 Oct 2026 00:28:46 UTC (1,061 KB)
来源：arXiv cs.AI · arxiv.org