arXiv cs.AI· Donghyun Lee, Taehoon Lee, Geonhee Ahn, Jieun Kim, Jihyun Park, Suyeon Cho, Yoona Kim, Chaerim Shin, Hoi Ri Moon, Jonggeol Na, Sukho Hong, Jihwan Oh, Soo Kyung Kim·· 12 小时前AI 评分27
MOF-Verify：面向 MOF 假设验证的失败感知智能体框架
MOF-VERIFY: A Failure-Aware Agentic Harness for MOF Hypothesis Verification
AI 导读
研究者提出诊断基准与 MOF-Verify 失败感知智能体框架，用于验证大语言模型驱动的材料科研流程。该基准含四类任务，覆盖结构锚定、合成条件验证、证据充分性验证与基于 MLIP 的计算验证，T-MOF-1-3 在闭卷、检索增强和 oracle 证据三种设置下定位失败环节。在多个基座 LLM 上，MOF-Verify 相较直接推理与检索基线提升了平均表现，增益集中在结构锚定与证据充分性。
正文 · 原文
Authors:Donghyun Lee, Taehoon Lee, Geonhee Ahn, Jieun Kim, Jihyun Park, Suyeon Cho, Yoona Kim, Chaerim Shin, Hoi Ri Moon, Jonggeol Na, Sukho Hong, Jihwan Oh, Soo Kyung Kim
Abstract:Large language models are increasingly used as reasoning components in AI-driven materials Co-Scientists, yet the reliability of the resulting verification pipeline remains unclear. Metal-organic frameworks (MOFs) provide a particularly challenging setting because structures may appear under different identifiers, synthesis outcomes depend strongly on experimental conditions, evidence is distributed across heterogeneous sources, and some hypotheses require computation rather than literature alone. We introduce a diagnostic benchmark with four task families covering structural grounding, synthesis-condition verification, evidence-sufficiency verification, and MLIP-based computational verification. T-MOF-1-3 are evaluated under closed-book, retrieval-enabled, and oracle-evidence settings to localize failures in knowledge access, evidence acquisition, and reasoning, while T-MOF-4 separately evaluates computational verification. Guided by these diagnosed failure modes, we develop MOF-Verify, a failure-aware agentic harness that targets structural, literature, evidence-sufficiency, and computational bottlenecks before producing a final verdict. Across multiple backbone LLMs, MOF-Verify improves the reported average over direct inference and retrieval-based baselines on this benchmark, with gains concentrated in structural grounding and evidence sufficiency. Benchmark records and release-status documentation are available at this https URL.
| Comments: | Accepted at the NeurIPS 2026 Workshops XAI4Science and AI4Mat. Revised manuscript with clarified methods, expanded supplementary information, and funding acknowledgements. 20 pages |
| Subjects: | Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE) |
| Cite as: | arXiv:2610.03056 [cs.AI] |
| (or arXiv:2610.03056v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03056 arXiv-issued DOI via DataCite |
Submission history
From: Dong Hyun Lee [view email]
[v1]
Fri, 2 Oct 2026 09:36:55 UTC (324 KB)
[v2]
Wed, 4 Nov 2026 20:49:07 UTC (324 KB)
来源：arXiv cs.AI · arxiv.org