arXiv cs.CL· Chanseul Cho, Noori Bae, Dongheon Lee, Youngmin Ro·· 11 小时前AI 评分31
GRIT：用梯度掩码纠正文本密集推理中的视觉误定位 token
GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning
AI 导读
针对视觉语言模型在文本密集推理中"答案正确但中间步骤误读图像"的 CVMR 现象，研究者提出训练框架 GRIT，将视觉定位信号引入 RLVR，通过梯度掩码抑制误定位 token 的影响。团队同时发布 GRIT-Bench，用于跨文本长度与推理深度系统评测。实验显示 GRIT 在推理准确率和定位忠实度上均显著优于现有 RLVR 方法，论文已被 WACV 2027 接收。
正文 · 原文
Abstract:Reinforcement learning with verifiable rewards (RLVR) has recently been extended to vision-language models (VLMs), yet its effectiveness remains limited for text-rich reasoning tasks. We highlight a key factor contributing to this limitation: models frequently arrive at correct final answers despite misperceiving visual information during intermediate steps. We define this phenomenon as Correct-but-Visually-Misgrounded Reasoning (CVMR). Because standard RLVR rewards only final-answer correctness, it fails to differentiate between properly grounded and misgrounded reasoning, reinforcing both equally. Through token-level analysis, we reveal that CVMR tokens exhibit distinct attention patterns that differ systematically from well-grounded ones. Based on this observation, we propose GRIT, a training framework that incorporates visual grounding into RLVR to mitigate the impact of CVMR. Furthermore, to address the lack of text-rich reasoning benchmarks that allow for systematic evaluation across text length and reasoning depth, we introduce GRIT-Bench. Experiments demonstrate that GRIT significantly improves both reasoning accuracy and grounding fidelity over existing RLVR methods.
| Comments: | Accepted to WACV 2027 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.41629 [cs.CV] |
| (or arXiv:2610.41629v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41629 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chanseul Cho [view email]
[v1]
Fri, 30 Oct 2026 12:05:54 UTC (4,250 KB)
来源：arXiv cs.CL · arxiv.org