arXiv cs.AI· Guijia Zhang, Hongbo Qiu, Jiajun Zha, Guohao Cai, Hao Zheng, Huihao Jing, Kexu Cheng, Shipeng Luo, JinYang Huang, Shing-Chi Cheung, Changsheng Lu, Haoxuan Che, Harry Yang·· 10 小时前AI 评分31
当图表与文字不符:评估科学论文中的自主视觉证据验证
When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers
AI 导读
研究在31篇完整论文上测试多模态模型能否自主发现图表与文字的矛盾,这些论文均含PRISMM-Bench中隐藏的图表-文字不一致。1445条开放式评审中60.6%提及问题图表,但仅3.3%被评分标准认定为诊断出矛盾;单独点名图表可将认定率从4.3%提升至20.4%,而一致性指令和粗粒度视觉提示无效。
正文 · 原文
Authors:Guijia Zhang, Hongbo Qiu, Jiajun Zha, Guohao Cai, Hao Zheng, Huihao Jing, Kexu Cheng, Shipeng Luo, JinYang Huang, Shing-Chi Cheung, Changsheng Lu, Haoxuan Che, Harry Yang
Abstract:Scientific figures carry evidence for the claims that cite them, so reviewing a paper means checking that a figure agrees with the text, and not only describing what the figure shows. Benchmarks of figure-text verification typically hand the model the figure, the claim, or the knowledge that an inconsistency exists, which leaves untested whether a model can find the figure worth checking in a complete paper, relate a visual element to the claim it conflicts with, and recognize the conflict. We study this autonomous setting on 31 full manuscripts, each carrying a reviewer-documented figure-text inconsistency from PRISMM-Bench that is hidden from the model. Six multimodal models review each PDF under a neutral prompt, and we then add one input at a time to the same manuscripts. Of 1,445 open-ended reviews, 60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction; a blind audit finds the rubric lenient, so we read these as rubric-accepted rates and compare conditions rather than absolute values. In a separate controlled run, naming the figure, without stating that it is flawed, raises rubric-accepted diagnosis from 4.3% to 20.4%, while a consistency instruction and coarse visual cues do not. When the contradiction is visible only in the rendered figure, text-only review yields no accepted diagnosis, while review of the rendered pages with the figure named reaches 15.8%. A baseline that lets the model choose which figures to check, ranked by centrality, stays at 4.8%. Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it. Multimodal document understanding should be evaluated on whether a model can propose what to verify, not only answer questions about a figure it is handed.
| Comments: | 28 pages, 5 figures, 14 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24353 [cs.AI] |
| (or arXiv:2611.24353v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24353 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hongbo Qiu [view email]
[v1]
Wed, 18 Nov 2026 12:47:35 UTC (1,468 KB)
来源:arXiv cs.AI · arxiv.org