arXiv cs.AI· Shashank Hegde, Alexander Popov, Elie Aljalbout, Nikolai Smolyanskiy·· 9 天前AI 评分21
LeWAM：基于扩散引导 MPC 的 JEPA 世界动作模型
LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC
AI 导读
LeWAM 是一个双向 Transformer 世界动作模型，在无解码器的 JEPA 潜空间上端到端训练，可同时完成前向、后向、逆动力学与策略预测。其潜空间在线性探针下读取机器人与物体状态优于 LeWM，且与同规模 flow-matching 策略的闭环表现相当。规划时在策略头的噪声空间采样，比直接采样原始动作更能提升闭环性能。
正文 · 原文
Abstract:World action models (WAMs) predict actions and future observations, typically from a reconstruction-based representation that carries noisy, redundant information which can complicate downstream predictions. We introduce LeWAM, a bidirectional transformer for forward, backward, inverse dynamics and policy prediction, on a decoder-free JEPA latent trained end-to-end through all four modes. We see the following benefits: 1) Alignment: linear probes read robot and object state from LeWAM's latent better than from a regular Le World Model (a forward-only JEPA world model), while the latent ignores visual distractors as well as LeWM does and far better than a reconstruction-based WAM. 2) Acting: Closed-loop evaluations of LeWAM match a regular flow-matching policy trained on the same encoder at matched size, while also providing a world model. 3) Planning: Sampling raw actions when planning with WAMs lets MPC exploit dynamics-model inaccuracies; planning in the noise space of the policy head instead improves the closed-loop performance of these WAMs.
| Comments: | 14 pages, 6 figures |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.12407 [cs.RO] |
| (or arXiv:2610.12407v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12407 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shashank Hegde [view email]
[v1]
Thu, 8 Oct 2026 17:46:03 UTC (2,366 KB)
来源：arXiv cs.AI · arxiv.org