arXiv cs.AI· Minghao Fu, Tavis Siebert, Eryk Halicki, Randall Balestriero·· 6 天前AI 评分39
LeWAM：用 JEPA 实现端到端世界模型与动作建模
LeWAM: End-to-End World and Action Modeling with JEPAs
AI 导读
LeWAM 是一个端到端 JEPA 控制模型，联合训练下一隐状态预测与动作流匹配，并对隐状态施加各向同性高斯约束，使单一模型既能直接执行策略，也能用学到的隐动力学评估采样动作进行规划。
正文 · 原文
Abstract:Joint-embedding predictive architectures (JEPAs) learn compact latent world models without pixel reconstruction. Existing JEPA pipelines typically train visual representations and action-conditioned dynamics before adding a control policy or solver. We introduce LeWAM, an end-to-end JEPA for control with: (1) a joint training objective over next latent state prediction and action flow matching, and (2) an isotropic Gaussian constraint on the latent state, enabling a single model to support direct policy execution and efficient planning by evaluating sampled actions with its learned latent dynamics. Compared with prior JEPAs in visual planning, LeWAM improves success rate from 28.6% to 89.7% on contact-rich tasks and from 10.9% to 46.2% on long-horizon tasks, and plans 32.3 times faster. On behavior cloning tasks in simulation and physical robots, LeWAM is also competitive with established methods. Interpretability analyses show that LeWAM learns control-relevant representations and action distributions effectively guiding test-time planning and control.
| Comments: | Project website: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.12818 [cs.RO] |
| (or arXiv:2610.12818v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12818 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Minghao Fu [view email]
[v1]
Thu, 8 Oct 2026 23:58:36 UTC (9,184 KB)
来源：arXiv cs.AI · arxiv.org