跳到正文
arXiv cs.AI· Li Li, Xiangbo Gao, Bo Ni, Samyadeep Basu, Viet Dac Lai, Jing Shi, Seunghyun Yoon, Gang Wu, Yu Wang, Thien Huu Nguyen, Tyler Derr, Franck Dernoncourt, Zhengzhong Tu, Yue Zhao·· 4 小时前AI 评分31

面向指令引导视频编辑的运动感知奖励模型

Motion-Aware Reward Modeling for Instruction-Guided Video Editing

AI 导读

针对指令引导视频编辑奖励模型偏好时序一致性而牺牲运动的问题(称为 motion collapse),研究者提出运动感知奖励模型,将 RGB 帧差流与针对运动抑制、伪造运动负样本的偏好训练结合,并用蒸馏锚点保持与人类质量判断的对齐。

正文 · 原文

Authors:Li Li, Xiangbo Gao, Bo Ni, Samyadeep Basu, Viet Dac Lai, Jing Shi, Seunghyun Yoon, Gang Wu, Yu Wang, Thien Huu Nguyen, Tyler Derr, Franck Dernoncourt, Zhengzhong Tu, Yue Zhao

View PDF HTML (experimental)

Abstract:Reward models for instruction-guided video editing can favor temporal consistency at the expense of motion. We study this failure, termed \emph{motion collapse}, using controlled motion-suppression variants of real edits. On VE-Bench, the evaluated composite reward prefers a motion-suppressed candidate in $97\%$ of cases. Learned video rewards and a VLM judge also struggle with mild suppression, while latent optimization and reward-guided fine-tuning exploit this weakness to increase reward as motion declines. A linear probe of our base reward backbone performs near chance on mild suppression, motivating an explicit inter-frame dynamics input. We propose a motion-aware reward model that combines an RGB frame-difference stream with preference training against motion-suppressed and fabricated-motion negatives. A distillation anchor preserves alignment with human quality judgments. On the VE-Bench motion ladder, our model reduces the rate of selecting motion-suppressed candidates to $0\%$ while achieving a Spearman correlation of $0.57$ with human quality ratings. It also detects unseen suppression variants and preserves motion during reward-guided fine-tuning on two editing pipelines. These results support explicitly modeling motion in rewards for video editing.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22495 [cs.CV]
  (or arXiv:2611.22495v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.22495

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Li Li [view email]
[v1] Tue, 17 Nov 2026 09:51:25 UTC (15,974 KB)

来源:arXiv cs.AI · arxiv.org