arXiv cs.CL· Yixuan Li, Yuxuan Jiang, Hanwei Zhu, Hancheng Zhu, Fan Zhang, Shiqi Wang, David Bull·· 14 小时前AI 评分26
从思维链到性能增益：强化学习如何改变基于推理的图像质量评估
Of Chains and Gains: What Reinforcement Learning changes on Reasoning-based Image Quality Assessment
AI 导读
研究通过采样感知评估、受控推理干预和捷径实验发现，RLVR 训练的 VLM 在大预算 max@k 评估下优于基座模型，但超出平均水平的提升集中在分布内。RLVR 生成的思维链移植到基座模型可提升性能，且固定思维链时评分也有改善，但推理时评分对思维链的依赖反而降低，7B 模型比 32B 更明显。RL 还能利用与 MOS 相关的文本线索提升评分准确率，可能虚高 benchmark 表现。
正文 · 原文
Abstract:Reinforcement Learning with Verifiable Rewards (RLVR) has improved the performance of large Vision-Language Models (VLMs) on image quality assessment (IQA). Existing approaches often attribute these gains to improved reasoning before scoring, supported by increasingly plausible Chain-of-Thought (CoT). However, whether these gains exceed sampling effects and how they depend on explicit reasoning remain unclear. We rigorously investigate these questions through sampling-aware evaluation, controlled reasoning interventions, and shortcut experiments. We find that RL-trained models outperform base models under large-budget $\operatorname{max}_g@k$ evaluation, while improvements beyond averaging are concentrated in-distribution. RLVR produces chains that raise base-model performance when transplanted and improves scoring with the chain held fixed. Yet scoring becomes less dependent on the chain during inference, more clearly at 7B than at 32B. Steering the probed quality direction in thinking tokens also has little effect on scoring. Finally, RL can improve scoring accuracy by exploiting textual cues correlated with mean opinion scores (MOSs), which could inflate benchmark performance. Together, these findings show that RLVR raises quality scoring and produces more useful chains, but that better scoring does not imply a stronger reliance on explicit reasoning, while score-based rewards can also encourage reliance on quality-irrelevant shortcuts. We hope this preliminary study can enlighten more and more concrete reasoning schemes in IQA.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.33203 [cs.CV] |
| (or arXiv:2610.33203v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33203 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yixuan Li [view email]
[v1]
Fri, 23 Oct 2026 18:49:15 UTC (176 KB)
来源：arXiv cs.CL · arxiv.org