arXiv cs.AI· Daniel J. Evans, J. P. Naiman, JooYoung Seo, J. Stephen Downie·· 3 天前AI 评分22
LMM 能否"看懂"多面板科学图表？一个面向大语言模型图表分析的多面板视觉问答数据集
Do you "see" what I "see"? A Multi-panel Visual Question Answer Dataset for Large Language Model Chart Analysis
AI 导读
研究者构建了一个面向 LLM/LMM 图表分析的多面板视觉问答数据集，用合成但逼真的单面板与多面板科学图表，按四级描述模型设计从概念问答到解释性分析的 VQA 任务。测试发现，随着面板数量、图表元素数量和问题难度增加，LMM 准确率下降；分析还引入成本相对性能作为指标。问答对、随机图表生成代码及图表元素 bounding-box 元数据均已公开。
正文 · 原文
Abstract:Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable, accessible, and reusable remains an open challenge. Large language and multimodal models (LLMs/LMMs) provide the possibility to automate many figure parsing tasks such as determining data trends or extracting data from plots. In this work, we consider if these models can reliably support task-oriented visual data analysis when presented with synthetic, but realistic, scientific figures that contain multiple charts. Our Visual Question Answering (VQA) process scaffolds our tasks along a four-level description model from basic conceptual answers to interpretive and analytical understanding. Additionally, we outline our process for synthetically generating our single and multi-panel (charts in which there are multiple panels present) plots and testing the data with various LMMs using quantitative analysis. In general, we find LMMs become less accurate with increasing figure complexity as the number of panels, number of plot elements, and difficulty of questions increase. Finally, because LMM accuracy is expected to change rapidly, our analysis also considers cost relative to performance as a metric to inform future user decisions. We provide the detailed question-answer pairs used in this analysis, along with the code used to generate random plots, question-answer pairs and detailed bounding-box metadata for chart elements. The code and data are available at this https URL and this https URL\_qa\_multipanel
| Comments: | 16 pages, 4 figures, accepted to The 30th International Conference on Theory and Practice of Digital Libraries, 2026 |
| Subjects: | Digital Libraries (cs.DL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2610.14173 [cs.DL] |
| (or arXiv:2610.14173v1 [cs.DL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14173 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jill Naiman [view email]
[v1]
Fri, 9 Oct 2026 18:00:02 UTC (272 KB)
来源：arXiv cs.AI · arxiv.org