arXiv cs.CL· Jiafeng Liang, Xianhao Ou, Tao Ren, Rongchuan Mu, Runxuan Liu, Dandan Tu, Ming Liu, See-Kiong Ng, Bing Qin·· 13 小时前AI 评分27
视频 RLVR 中的捷径放大：从感知不确定性坍缩视角揭示并推出 BSPO
Uncovering Shortcut Amplification in Video RLVR through the Lens of Perceptual Uncertainty Collapse
AI 导读
研究发现视频 RLVR 的结果级优化会放大捷径效应，模型更倾向文本先验答案而非视觉证据，并识别出“感知不确定性坍缩”现象：RL 训练持续锐化答案分布，但视觉证据带来的不确定性下降逐步减弱。
正文 · 原文
Abstract:Reinforcement learning with verifiable rewards (RLVR) has emerged as an effective post-training paradigm for enhancing the video reasoning capability of Large Multimodal Models (LMMs). Despite its empirical effectiveness, we show that outcome-level optimization can introduce an undesirable shortcut amplification effect, whereby models increasingly favor text-prior answers even when they are not supported by the visual evidence. To characterize this behavior, we investigate the training dynamics of video RLVR through the lens of perceptual uncertainty, analyzing how predictive uncertainty evolves under video-conditioned and text-only contexts. Our analysis identifies a perceptual uncertainty collapse phenomenon, in which RL training continuously sharpens the model's answer distribution while the uncertainty reduction attributable to visual evidence progressively diminishes. A further gradient-direction analysis reveals that such confidence enhancement becomes increasingly less aligned with visually grounded uncertainty reduction and is instead dominated by text-prior compression. Motivated by these findings, we propose Bifocal Self-distillation Policy Optimization (BSPO), a self-distillation framework that regulates policy updates with two complementary distributional signals. Specifically, BSPO derives empirical reasoning principles from intra-group correct and incorrect rollouts to identify reasoning-sensitive tokens, and introduces shortcut-aware visual perturbation to penalize predictions that remain invariant when task-relevant visual evidence is weakened. By adaptively reweighting the original RL advantage, BSPO preserves the task-solving benefits of RLVR while promoting updates that are both reasoning-sensitive and visually grounded. Extensive experiments validate the effectiveness of BSPO in improving video reasoning performance and strengthening visual faithfulness.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.37211 [cs.CV] |
| (or arXiv:2610.37211v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37211 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiafeng Liang [view email]
[v1]
Tue, 27 Oct 2026 13:26:13 UTC (1,200 KB)
来源：arXiv cs.CL · arxiv.org