arXiv cs.CL· Omar Sharif, Eftekhar Hossain, Nikhil Singh, Patrick Ng·· 12 小时前AI 评分34
通过奖励设计分离多模态 LLM 的感知与推理能力
Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design
AI 导读
研究在算法视觉谜题上发现，多模态模型的瓶颈是感知而非推理：将图像替换为简单文本描述后，Claude 模型平均性能提升超过 20 分。用 GRPO 训练 Qwen-2.5-VL-7B 时，奖励设计能诱导带自我反思和视觉引用的长结构化推理，较基座模型提升 5.56 分，但收益不均衡，没有单一奖励能改善所有类别。
正文 · 原文
Abstract:Reinforcement learning with verifiable rewards has driven major gains in LLM reasoning, and it is intuitive to assume this recipe will transfer well to multimodal models. However, multimodal models do two things: first, perceive what is in an image, then reason about what it implies. Because these stages are graded jointly, it is hard to tell how much room reasoning alone has to grow. We study this on algorithmic visual puzzles, where both components are necessary and show that perception, not reasoning, is the binding constraint. Replacing images with simple textual descriptions raises performance by over 20 points on average for Claude models. We then evaluate six reward designs aimed at inducing visual grounding during reasoning without chain-of-thought supervision. Training Qwen-2.5-VL-7B with GRPO, reward design induces long, structured reasoning with self-reflection and visual references, yielding a 5.56-point gain over the base model. These gains are, however, uneven; no single reward improves all categories, and rewards with verifiable accuracy signals trade cross-domain transfer for in-domain accuracy. These results point to perception-aware reward design as a path forward, so that signals correct perception at its source rather than the reasoning that inherits its errors.
| Comments: | Accepted at AACL-ICJNLP |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2601.00215 [cs.CV] |
| (or arXiv:2601.00215v3 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2601.00215 arXiv-issued DOI via DataCite |
Submission history
From: Omar Sharif [view email]
[v1]
Thu, 1 Jan 2026 05:19:28 UTC (8,918 KB)
[v2]
Mon, 15 Jun 2026 20:37:49 UTC (9,307 KB)
[v3]
Sat, 31 Oct 2026 19:08:55 UTC (9,304 KB)
来源：arXiv cs.CL · arxiv.org