arXiv cs.AI· Xicheng Gong, Wenxuan Song, Chengze Liu, Qiwei Li, Qize Yu, Jiaolong Yang, Haoang Li, Yadong Mu·· 12 小时前AI 评分12
JADE：让 VLM 与动作专家联合去噪的机器人动作生成方法
Vision-Language Models Are Joint Denoisers in Robotic Action Generation
AI 导读
JADE 提出让 VLM 与扩散动作专家联合去噪，VLM 在每一步读取带噪动作、视觉观测和语言指令，并通过门控交叉注意力引导动作专家，使预训练感知与推理持续参与动作生成。
正文 · 原文
This paper has been withdrawn by Xicheng Gong
No PDF available, click to view other formats
Abstract:Diffusion-based vision-language-action (VLA) models cascade pretrained vision-language models (VLMs) with diffusion action experts, but typically restrict the VLM to providing static conditioning. This separation limits how much pretrained perception and reasoning can guide action generation, especially in out-of-distribution settings. We introduce JADE, a Joint Action DEnoiser that enables the VLM and action expert to jointly refine actions. Our key insight is that structured action latents allow the VLM to read the current noisy action together with visual observations and language instructions. At every denoising step, the VLM contextualizes the noisy action and guides the action expert through gated cross-attention, so that its guidance evolves with the action being generated. We further supervise the VLM to recover clean action latents from noisy ones, which gives it a direct action-learning signal and couples perception with control. Additionally, joint denoising enables JADE to generate actions with fewer sampling steps, helping offset the additional cost of VLM participation. An adaptive step-size schedule further balances task performance and inference latency. On RoboDojo, JADE achieves an average success rate of 12.67%, the highest among the evaluated $\pi_{0.5}$-based methods, and is competitive with recent VLA and WAM foundation models pretrained at scale. It also improves performance on RoboTwin 2.0 and raises the average success rate across four real-world bimanual tasks by 20.8 percentage points compared with $\pi_{0.5}$.
| Comments: | formatting error |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2611.01215 [cs.RO] |
| (or arXiv:2611.01215v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01215 arXiv-issued DOI via DataCite |
Submission history
From: Xicheng Gong [view email]
[v1]
Sat, 31 Oct 2026 19:29:39 UTC (3,861 KB)
[v2]
Thu, 5 Nov 2026 05:41:38 UTC (1 KB) (withdrawn)
来源：arXiv cs.AI · arxiv.org