跳到正文
arXiv cs.AI· Lixuan He, Tianzhe Chu, Shulin Tian, Shikang Zheng, Feng Chen, Yi Ma·· 4 小时前AI 评分31

Gaze-RL:从多模态推理内部计算中推断奖励落点

Eyes in the Update? Learning Where Rewards Land in Multimodal Reasoning

AI 导读

Gaze-RL 提出一种表征驱动的策略优化方法,通过潜在奖励落点场重新分配轨迹级优势,该场由模型内部跨模态几何与局部预测不确定性计算得出,对视觉锚定且影响决策的位置赋予更大更新权重,同时保持均值归一以保留每条 rollout 的总信用。方法无需过程监督、辅助感知模块或架构修改,在多项图像与视频基准上同时提升 Qwen2.5-VL 和 InternVL3.5。

正文 · 原文

View PDF HTML (experimental)

Abstract:Multimodal post-training has moved from broad instruction tuning toward reinforcement learning for image and video reasoning, where outcome rewards can improve final-answer accuracy. Yet these rewards are assigned to complete generated trajectories, while visual evidence is used only at sparse and uneven positions inside the response. Recent work has begun to make multimodal RL more token-sensitive through grounding signals or external token-level weighting. We ask a more specific question: can the allocation of the same trajectory-level reward be inferred from the model's own multimodal computation, rather than from an auxiliary annotator, perception module, or process label? We propose Gaze-RL, a representation-driven policy optimization method that redistributes completion-level advantage through a latent reward landing field. The field is computed from internal cross-modal geometry and local predictive uncertainty, assigning larger update weight to positions that are both visually anchored and decision-bearing while remaining mean-one, so the total credit of each rollout is preserved and only its positionwise allocation changes. Gaze-RL requires no process supervision, auxiliary perception module, or architectural modification, and improves both Qwen2.5-VL and InternVL3.5 across a broad range of image and video benchmarks.
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22438 [cs.LG]
  (or arXiv:2611.22438v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.22438

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Lixuan He [view email]
[v1] Tue, 17 Nov 2026 09:11:42 UTC (8,538 KB)

来源:arXiv cs.AI · arxiv.org