arXiv cs.CL· Shuai Dong, Kanghui Tian, Linghan Chen, Wenqi Shao, Hongrui Wu, Haowen Hou, Yongfu Zhu, Xuehui Wang, Ziyue Wang, Wenpu Liu, Weichu Xie, Yuqi Xu, Congcong Wang, Kaiwen Tuo, Caoyuan Ma, Taiqiang Wu, Ji Zhao, Wenzheng Chang, Dianyi Wang, Siyuan Wang, Zhongyu Wei, Yu Qiao, Jiaqi Wang·· 12 小时前AI 评分31
TurnWise：用轮级信用分配改进流式视频 RL
Turns Matter: Advancing Streaming Video RL via Turn-Level Credit
AI 导读
研究提出 TurnWise，一个轮级 GRPO 框架，用于流式视频理解中模型反复决定"回应还是继续观察"的场景。它用 silence-completed prefix scoring 从每次决策的剩余回报构造组相对优势，并以轮内、轮间平均的 token 损失让回应与等待决策等权。实验显示该方法优于此前的流式视频理解 RL 方法，且无需学习 critic 或额外 rollout。
正文 · 原文
Authors:Shuai Dong, Kanghui Tian, Linghan Chen, Wenqi Shao, Hongrui Wu, Haowen Hou, Yongfu Zhu, Xuehui Wang, Ziyue Wang, Wenpu Liu, Weichu Xie, Yuqi Xu, Congcong Wang, Kaiwen Tuo, Caoyuan Ma, Taiqiang Wu, Ji Zhao, Wenzheng Chang, Dianyi Wang, Siyuan Wang, Zhongyu Wei, Yu Qiao, Jiaqi Wang
Abstract:Streaming video understanding requires models to repeatedly decide whether to respond or continue observing. Recently, Group Relative Policy Optimization (GRPO) has been applied to this setting, with existing approaches emphasizing reward design while retaining a single trajectory-level advantage across turns. Such shared credit can reinforce redundant replies after earlier successes and discourage useful follow-ups after earlier failures. We propose TurnWise, a turn-level GRPO framework that aligns learning with individual streaming decisions. TurnWise addresses the mismatch between trajectory-level feedback and turn-level interaction, promoting informative responses and reducing redundancy. Specifically, turn-level credit assignment uses silence-completed prefix scoring to construct group-relative advantages from each decision's remaining return, separating its learning signal from earlier rewards. Turn-mean policy optimization averages token losses within each turn and then across turns, giving response and waiting decisions equal weight regardless of output length. These complementary designs improve streaming policy learning without a learned critic or additional rollouts. Extensive experiments show our method outperforms prior RL approaches for streaming video understanding.
| Comments: | 16 pages, 4 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.06728 [cs.CV] |
| (or arXiv:2611.06728v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06728 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shuai Dong [view email]
[v1]
Wed, 4 Nov 2026 16:16:27 UTC (10,730 KB)
来源：arXiv cs.CL · arxiv.org