arXiv cs.AI· Zhongan Bi, Kepeng Lin, Xuanang Gao, Yuhan Sun, Lianrun Zhang·· 3 天前AI 评分29
视觉敏感性不等于主张可撤回：多模态强化学习中的持久性感知信用分配
Visual sensitivity is not claim retractability: persistence-aware credit assignment for multimodal reinforcement learning
AI 导读
针对 RLVR 训练中视觉主张缺乏图像支撑的问题，研究者提出持久性感知信用门控（PACG），对异常持久的视觉主张衰减正向信用，无需标注且不增加推理成本。在 Qwen2.5-VL-7B 上，PACG 将九项基准三种子均值从 58.1% 提升至 59.9%（DAPO）、从 59.8% 提升至 60.9%（VPPO），并提升 HallusionBench 准确率。
正文 · 原文
Abstract:Reinforcement Learning with Verifiable Rewards (RLVR) has been extended to Large Vision-Language Models (LVLMs), and perception-aware methods further encourage policies to rely on visual evidence. Yet relying on the image does not guarantee that visual claims are supported by it. Before RL training, 27.81% of the correctly answered responses of Qwen2.5-VL-7B on four multimodal reasoning benchmarks contain at least one direct visual claim that the image does not support. Since outcome-level RL rewards each response as a whole, these claims inherit the positive credit of the correct answer. We introduce a fixed-rollout counterfactual diagnostic that re-scores the same response under an intervened image to separate Evidence-Function Sensitivity (EFS), how strongly the model's predictions change, from claim persistence, whether the model keeps supporting the same claim rather than retracting it. The diagnostic reveals Sensitivity-Persistence Decoupling (SPD): under DAPO and VPPO, EFS increases and claims become more retractable overall, yet unsupported claims become significantly more persistent, whereas GRPO raises EFS without this deterioration. We therefore propose Persistence-Aware Credit Gating (PACG), which attenuates positive credit for unusually persistent visual claims and leaves all other credit unchanged. It requires no supported/unsupported labels and adds no inference cost. On Qwen2.5-VL-7B, PACG raises the nine-benchmark average over three seeds from 58.1% to 59.9% with DAPO and from 59.8% to 60.9% with VPPO, while making unsupported claims more retractable. The gains extend to a larger model, a newer backbone, and the accuracy of HallusionBench also improves consistently. These results suggest that visual sensitivity and claim retractability are complementary dimensions of multimodal credit assignment.
| Comments: | MLLM,RL |
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | F.2.2; I.2.7 |
| Cite as: | arXiv:2609.36572 [cs.AI] |
| (or arXiv:2609.36572v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.36572 arXiv-issued DOI via DataCite |
Submission history
From: Zhongan Bi Xinfei [view email]
[v1]
Tue, 29 Sep 2026 02:52:07 UTC (3,825 KB)
[v2]
Wed, 30 Sep 2026 02:54:06 UTC (3,824 KB)
[v3]
Sun, 11 Oct 2026 15:00:28 UTC (3,824 KB)
来源：arXiv cs.AI · arxiv.org