arXiv cs.CL· Andrei-Alexandru Manea, Jind\v{r}ich Libovick\'y, Philipp J. R\"osch·· 11 小时前AI 评分31
MultiInpaintCOCO：多语言细粒度视觉理解基准发布
MultiInpaintCOCO: Fine-Grained Visual Understanding in Multiple Languages
AI 导读
研究者将英文细粒度视觉数据集 InpaintCOCO 扩展至覆盖五大语系的 13 种语言，构建出含 953 个最小对比样本的多语言基准 MultiInpaintCOCO。测试显示低资源语言性能显著下降，最强预测因素是语言可用数据资源而非与英语的类型学相似度；跨语言检索中多数模型存在偏向英语的系统性偏差，Gemma 3 英语偏好最强，Qwen 3 偏差最小，尽管两者单语准确率相近。
正文 · 原文
Abstract:Previous research shows that vision-language (VL) models struggle to understand fine-grained image details and tend to focus on the overall layout. Moreover, any weakness in English tends to be further amplified in the multilingual setting. To study that, we extend InpaintCOCO, an English dataset focused on fine-grained visual details, to 13 typologically diverse languages across five language families, creating MultiInpaintCOCO - a new multilingual benchmark of 953 minimal-pair examples. We further test several state-of-the-art models in multilingual multimodal retrieval, considering both light VL CLIP-like encoders and large language models. Our results show that performance degrades substantially for lower-resource languages, with the strongest predictor being the data resources available for the languages rather than typological similarity to English. In cross-lingual retrieval, most models exhibit a systematic bias toward English: when given a correct caption in a non-English language and an incorrect one in English, models preferentially select the English caption. This bias varies considerably across architectures. Among the tested models, Gemma 3 shows the strongest English preference while Qwen 3 is the least biased, despite both reaching similar monolingual accuracy across languages. The code is available at this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Report number: | 45 |
| Cite as: | arXiv:2611.03309 [cs.CL] |
| (or arXiv:2611.03309v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.03309 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Proceedings of the 6th Workshop on Multilingual Representation Learning (MRL 2026), pages 560-585 |
| Related DOI: | https://doi.org/10.18653/2026.mrl-1.45
DOI(s) linking to related resources |
Submission history
From: Andrei-Alexandru Manea [view email]
[v1]
Mon, 2 Nov 2026 15:39:35 UTC (18,466 KB)
来源：arXiv cs.CL · arxiv.org