arXiv cs.AI· Taeyoung Kim, Junyoung Sung, Heeseung Kwon, Minseop Kim, Jinwoo Shin·· 13 小时前AI 评分32
MoGAP：用运动引导的非对称预测让视频模型适配机器人控制
Adapting Video Models for Robot Control via Motion-guided Asymmetric Prediction
AI 导读
MoGAP 通过运动引导的非对称预测，将视觉监督集中在动态区域，在共享视频扩散骨干内以仅训练用的辅助预测路径与标准视频-动作预测联合训练，无需外部教师模型或额外模态。在 GR-1 桌面基准上，Fast-WAM（Wan2.2-5B）成功率从 38.8% 提升至 47.5%，Cosmos3-Nano 从 52.3% 提升至 62.3%。
正文 · 原文
Abstract:World Action Models (WAMs) leverage pretrained video generative models for robotic control by jointly predicting future visual observations and actions. The visual prediction objective plays a key role in adapting pretrained video representations to the target robot domain. Yet standard visual prediction inherited from video pretraining treats all visual regions uniformly, despite their differing relevance to robotic control. We therefore investigate the spatial allocation of visual supervision in WAMs and show that focusing supervision on dynamic regions can substantially improve policy performance. Building on this finding, we introduce Motion-Guided Asymmetric Prediction (MoGAP), which augments WAM training with motion-focused visual prediction. MoGAP allocates noise asymmetrically across regions, training the model to reconstruct more heavily corrupted dynamic regions using less corrupted static regions as context. MoGAP is trained alongside standard video-action prediction through a training-only auxiliary prediction path within the shared video diffusion backbone. It uses motion cues derived directly from training videos, requiring neither an external teacher nor additional modalities. Across robot manipulation benchmarks and WAM architectures, MoGAP consistently improves policy performance, increasing success rates on the GR-1 tabletop benchmark from 38.8% to 47.5% for Fast-WAM (Wan2.2-5B) and from 52.3% to 62.3% for Cosmos3-Nano.
| Comments: | Project page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36624 [cs.CV] |
| (or arXiv:2610.36624v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36624 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Taeyoung Kim [view email]
[v1]
Tue, 27 Oct 2026 04:27:20 UTC (2,929 KB)
来源：arXiv cs.AI · arxiv.org