arXiv cs.AI· Aotian Chen, Frazier N. Baker, Reza Averly, Shan Zhang, Tianao Chen, Daniel Adu-Ampratwum, Xia Ning, Yuan Xue·· 3 天前AI 评分26
ChemExplain：面向解释性化学推理的多模态基准
ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning
AI 导读
研究者推出 ChemExplain，一个从 383 篇同行评审论文中构建的 1180 道多选题化学基准，每题给出分子或反应图像、观测到的化学效应和四个候选解释，要求模型完成"解释性归因"。基准由多智能体框架自动构建并经八位化学领域专家验证，最强模型准确率为 78.6%；在 600 题专家复核子集上，该模型 71.4% 的错误被归类为归因失败。
正文 · 原文
Abstract:Current chemistry benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an observed chemical effect occurs. We introduce ChemExplain, a benchmark of 1,180 multiple-choice chemistry questions drawn from 383 peer-reviewed papers. Each question pairs a molecular or reaction image with an observed chemical effect and four candidate explanations, each associated with a different structural feature visible in the image. Answering requires the model to perceive the depicted structure, apply chemical knowledge, and identify which feature most directly explains the effect, a capability we term explanatory attribution. The correct feature-mechanism relation is derived from supporting evidence in the source paper rather than invented by the question-generation LLM. We construct the benchmark automatically with a multi-agent framework that applies progressive quality control at three stages and validate it with eight chemistry domain experts. The strongest evaluated model reaches 78.6% accuracy, leaving considerable room for improvement. On a 600-question expert-reviewed subset, reasoning-trace analysis classifies 71.4% of the strongest model's errors as attribution failures. Attribution is also the largest error category for the open-source and chemistry-specific models included in our analysis.
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.17183 [cs.AI] |
| (or arXiv:2610.17183v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17183 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuan Xue [view email]
[v1]
Mon, 12 Oct 2026 17:53:56 UTC (4,960 KB)
来源：arXiv cs.AI · arxiv.org