arXiv cs.AI· Zuyao Lin, Jianhui Zhang, Peidong Jia, Xiaoguang Zhao, Shanghang Zhang, Jingdong Wang, Xingyu Chen·· 9 天前AI 评分24
World-Ego Model（WEM）：面向长时程导航-操作任务的具身视频生成世界-自我建模
World-Ego Modeling for Embodied Video Generation in Long-Horizon Navigation-Manipulation Tasks
AI 导读
研究者提出 World-Ego Model（WEM），将具身视频拆解为「世界」（背景与未被操作物体）和「自我」（机器人与被操作物体）并解耦生成过程，结合角色条件注意力（RCA）的视觉-语言状态预测器与语义路由 MoE（SR-MoE）扩散生成器。
正文 · 原文
Abstract:Embodied video world models typically capture both scene evolution and the robot's behavior, which we refer to as the \emph{world} and the \emph{ego}, respectively. The world and the ego exhibit different underlying dynamics: world prediction relies primarily on visual history and emphasizes scene stability, whereas ego prediction relies more strongly on the current instruction and emphasizes accurate instruction following. Modeling both components within a single generation stream can entangle these different dependencies, making it difficult to specialize the prediction of either component. Consequently, it becomes difficult to simultaneously maintain scene consistency and accurate instruction following, particularly in long-horizon navigation-manipulation tasks. In this paper, we propose to decompose an embodied video into the world and the ego and disentangle their generation processes. Specifically, we define the world as the background and currently unmanipulated objects, and the ego as the robot and currently manipulated objects. Based on this definition, we develop the \emph{World-Ego Model} (WEM), which combines a vision-language state predictor using role-conditioned attention (RCA) and asymmetric query budgets with a semantic-routed mixture-of-experts (SR-MoE) diffusion generator. To enable rigorous evaluation, we further construct HTEWorld, a dataset and benchmark for long-horizon embodied video generation with hybrid navigation-manipulation tasks, providing 125K training video clips comprising over 4.5M frames with fine-grained instructions, together with 300 multi-turn evaluation trajectories covering over 2K instructions. Extensive experiments show that WEM achieves state-of-the-art performance on HTEWorld while remaining competitive on existing manipulation-oriented evaluations.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2605.19957 [cs.CV] |
| (or arXiv:2605.19957v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19957 arXiv-issued DOI via DataCite |
Submission history
From: Zuyao Lin [view email]
[v1]
Tue, 19 May 2026 15:10:27 UTC (5,620 KB)
[v2]
Thu, 8 Oct 2026 14:46:51 UTC (5,501 KB)
来源：arXiv cs.AI · arxiv.org