arXiv cs.CL· Jiwan Chung, Neel Joshi, Pratyusha Sharma, Youngjae Yu, Vibhav Vineet·· 12 小时前AI 评分40
MathLens 基准揭示 MLLM 多模态推理能力由感知与推理子技能构成
What MLLMs Learn about When they Learn about Multimodal Reasoning
AI 导读
研究者提出 MathLens 基准，将多模态推理拆解为感知、推理和多模态特有成分，用教材式几何题配合纯文本变体与感知探针分别测量。结果显示强化学习主要提升感知 grounding 与图表变化鲁棒性，文本 SFT 的增益则来自反思式推理；随着感知与推理改善，剩余错误中多模态特有成分占比上升。作者据此认为多模态推理的进展是子技能间平衡的转移，而非整体能力提升，呼吁超越单一准确率评分。
正文 · 原文
Abstract:Evaluation of multimodal reasoning models is typically reduced to a single accuracy score, implicitly treating reasoning as a unitary capability. We introduce MathLens, a benchmark of textbook-style geometry problems that exposes this assumption by operationally decomposing performance into perception, reasoning, and multimodal-specific components. Each problem is derived from a symbolic specification and accompanied by visual diagrams, text-only variants, multimodal questions, and targeted perceptual probes, enabling controlled measurement of each component. Using this decomposition, we show that common training strategies are associated with systematically different capability profiles that are invisible under aggregate accuracy. Across the models we study, reinforcement learning primarily improves perceptual grounding and robustness to diagram variation, while our analysis suggests that textual SFT yields gains through reflective reasoning. In contrast, as perception and reasoning improve, a growing fraction of remaining errors fall outside these components and are categorized as multimodal-specific. These results suggest that apparent progress in multimodal reasoning reflects shifting balances among subskills rather than uniform advancement, motivating evaluation beyond scalar accuracy.
| Comments: | NeurIPS 2026. Project page: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2510.01719 [cs.CL] |
| (or arXiv:2510.01719v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2510.01719 arXiv-issued DOI via DataCite |
Submission history
From: Jiwan Chung [view email]
[v1]
Thu, 2 Oct 2025 06:58:29 UTC (10,710 KB)
[v2]
Tue, 7 Oct 2025 15:17:04 UTC (10,710 KB)
[v3]
Tue, 3 Feb 2026 08:57:11 UTC (10,984 KB)
[v4]
Thu, 7 May 2026 09:13:59 UTC (10,973 KB)
[v5]
Fri, 23 Oct 2026 02:22:08 UTC (10,998 KB)
来源：arXiv cs.CL · arxiv.org