arXiv cs.CL· Donghwan Kim·· 6 天前AI 评分44
LLM 作为工业故障诊断工具上层集成层时为何未能实现互补优势
When LLMs Sit Above Diagnostic Tools: Unrealized Complementarity in Industrial Fault Diagnosis
AI 导读
研究在五个诊断数据集（轴承振动、过程监控、半导体设备）上测试 LLM 能否可靠利用外部诊断信息，结果所有数据集中冲突的外部信息都推翻了 LLM 原本正确的判断。
正文 · 原文
Abstract:Large language models are increasingly used as integration layers above specialized tools, but a stronger component does not necessarily produce a stronger combined system. Across five diagnostic datasets (bearing vibration, process monitoring, semiconductor equipment), we study whether an LLM can reliably use external diagnostic information; paired repeat calls separate advice effects from output instability. In all five, conflicting external information overturned initially correct LLM judgments. Among the four datasets with direct integration comparisons, none showed a consistent advantage for implicit LLM integration over the stronger standalone source. On a Tennessee Eastman confirmation set whose protocol was fixed before evaluation, unaided accuracy was 64.67%, implicit LLM-specialist integration 77.43%, and the specialist alone 83.33%. Specialist information improved the LLM by 12.8 points (95% interval 9.7 to 15.9), yet the integrated output stayed 5.9 points below the specialist (95% interval -12.0 to -0.7). A two-source selector oracle reached 92.76%, indicating complementarity that the integrated output did not fully realize. The integrated output missed 140 of 295 specialist corrections (47.5%) but lost 15 of 99 initially correct LLM judgments (15.2%). The deficit remained under prompt and specialist sensitivity analyses. Among CWRU cases solved under both evidence presentations, task-aligned physical evidence yielded lower estimates of susceptibility to incorrect advice in six of seven models (five intervals excluding zero); higher reasoning effort gave no reliable reduction in five models, and a separate four-model TEP analysis gave no clear evidence that it resolves the integration problem. Source quality and integration quality should be evaluated separately: an integration layer should be compared with its stronger standalone component, not only with the unaided LLM.
| Comments: | 34 pages, 5 figures + 1 appendix figure |
| Subjects: | Computation and Language (cs.CL); Systems and Control (eess.SY) |
| Cite as: | arXiv:2610.05031 [cs.CL] |
| (or arXiv:2610.05031v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.05031 arXiv-issued DOI via DataCite |
Submission history
From: Donghwan Kim [view email]
[v1]
Sun, 4 Oct 2026 07:51:31 UTC (289 KB)
[v2]
Thu, 8 Oct 2026 18:20:18 UTC (275 KB)
来源：arXiv cs.CL · arxiv.org