arXiv cs.AI· Zhehao Dong, Xiaofeng Wang, Zheng Zhu, Yirui Wang, Yang Wang, Yukun Zhou, Boyuan Wang, Chaojun Ni, Runqi Ouyang, Wenkang Qin, Xinze Chen, Yun Ye, Guan Huang, Zhen Lu, Yue Yang·· 12 小时前AI 评分38
EMMA：用生成式视觉模型提升机器人控制鲁棒性
EMMA: Enhancing Robotic Control Robustness through a Generative Visual Model
AI 导读
研究团队提出 EMMA 框架，通过生成式数据引擎与训练流程增强 VLA 策略，其 DreamTransfer 基于扩散 Transformer 生成多视角一致、几何合理的机器人操作视频，支持对前景、背景和光照的文本驱动编辑。在零样本视觉设置的真实机器人任务中，该框架相比仅用真实数据训练实现超 92% 的相对性能提升，加入 AdaMix 自适应样本加权策略后再提升 17%。
正文 · 原文
Authors:Zhehao Dong, Xiaofeng Wang, Zheng Zhu, Yirui Wang, Yang Wang, Yukun Zhou, Boyuan Wang, Chaojun Ni, Runqi Ouyang, Wenkang Qin, Xinze Chen, Yun Ye, Guan Huang, Zhen Lu, Yue Yang
Abstract:The generalization of vision-language-action (VLA) models heavily relies on diverse training data. However, acquiring large-scale robot manipulation data across varied object appearances is costly and labor-intensive. To address this, we introduce Embodied Manipulation Media Adaptation (EMMA), a framework for augmenting VLA policies that combines a generative data engine with an effective training pipeline. We introduce DreamTransfer, a diffusion Transformer-based architecture for generating multi-view consistent and geometrically plausible embodied manipulation videos. It supports text-driven video editing on foreground, background and lighting while preserving 3D structure and geometric fidelity. We also utilize a hybrid training set of real and generated data and propose AdaMix to enhance the training process. AdaMix is a training strategy that adaptively weights samples according to policy performance to emphasize challenging samples. Evaluations show DreamTransfer outperforms previous video generation models in multi-view consistency, geometric fidelity, and text-conditioning precision. In real-world robotic tasks with zero-shot visual settings, our framework achieves a relative performance increase of over 92% compared to training with real data alone, and improves by an additional 17% with AdaMix.
| Subjects: | Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2509.22407 [cs.AI] |
| (or arXiv:2509.22407v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2509.22407 arXiv-issued DOI via DataCite |
Submission history
From: Zhehao Dong [view email]
[v1]
Fri, 26 Sep 2025 14:34:44 UTC (3,654 KB)
[v2]
Mon, 16 Mar 2026 09:11:10 UTC (25,105 KB)
[v3]
Tue, 20 Oct 2026 17:33:39 UTC (28,348 KB)
来源：arXiv cs.AI · arxiv.org