arXiv cs.CL· Changjiang Jiang, Jiang Zhou, Xinbin Yuan, Fan Zhang, Tao Feng, Zhuohan Xie·· 9 小时前AI 评分31
MInfoVisQA:面向信息密集型视觉问答的多语言基准
MInfoVisQA: A Multilingual Benchmark for Information-Rich Visual Question Answering
AI 导读
研究者提出 MInfoVisQA,一个覆盖 24 种语言、由 128 个种子实例扩展出 8,960 个 QA 配置的平行基准,用于检验多模态大模型在表层语言变化时对信息密集型视觉内容的理解一致性。
正文 · 原文
Abstract:Multimodal large language models (MLLMs) excel in vision-language tasks; yet, whether they maintain consistent understanding of information-rich visuals (InfoVis) when the surface language changes remains unclear. To quantify this capability, we introduce MInfoVisQA, a parallel benchmark with 8,960 QA configurations derived from 128 seed instances across 24 languages. We construct seed-matched, language-parallel visuals and QA pairs by disentangling numerical-structural data from text and programmatically synchronizing translated labels. We evaluate 25 MLLMs under two settings: Language-Aligned QA (LQA, matching visual-text languages) and Cross-Language QA (XQA, fixed QA language with varying visual languages). The results show that aggregate accuracy masks substantial robustness gaps: two models with comparable mean LQA accuracy (80.4\%) exhibit performance spreads of 7.8 and 20.3 percentage points across languages. Moreover, all 25 models score lower with non-English visuals under fixed English QA, underscoring the need for cross-language sensitivity metrics.
| Comments: | Dataset: this https URL Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2611.21644 [cs.CL] |
| (or arXiv:2611.21644v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21644 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Changjiang Jiang [view email]
[v1]
Wed, 14 Oct 2026 10:45:36 UTC (19,208 KB)
来源:arXiv cs.CL · arxiv.org