arXiv cs.AI· Zimu Han, Yiming Zeng, Jiyao Zhang, Zihao Zhao, Yuanfei Wang, Yixiang Jin, Shiqi Li, Shuangben Chen, Wei Huang, Ruodai Li, Hui Shen, Hao Dong·· 10 天前AI 评分28
HIL-UMI：将 VLA 模型人在回路后训练带入通用操作接口
HIL-UMI: Bringing Human-in-the-Loop Post-Training of Vision-Language-Action Models to Universal Manipulation Interface
AI 导读
HIL-UMI 是一个基于通用操作接口（UMI）的免机器人人在回路 VLA 后训练框架，可在手持演示时查询当前策略而不执行其预测。它用 Energy Score 对比人类动作轨迹与策略推理，在差异指示分布外状态时触发数据采集，并通过在线优势预测筛选关键片段来改进基于进度的优势估计器。
正文 · 原文
Authors:Zimu Han, Yiming Zeng, Jiyao Zhang, Zihao Zhao, Yuanfei Wang, Yixiang Jin, Shiqi Li, Shuangben Chen, Wei Huang, Ruodai Li, Hui Shen, Hao Dong
Abstract:Large-scale vision-language-action (VLA) models provide powerful priors for robot manipulation, yet adapting them to a specific deployment remains challenging. Supervised fine-tuning (SFT) on task-specific demonstrations provides a step toward deployment, but faces two persistent limitations: static data provide limited coverage of out-of-distribution states, and standard imitation objectives do not distinguish progressing behavior from less useful data. Interactive post-training can address these limitations, but typically requires repeated policy execution and human intervention on a physical robot. We introduce HIL-UMI, a policy-guided Universal Manipulation Interface (UMI) framework for robot-free human-in-the-loop VLA post-training. During handheld UMI demonstrations, HIL-UMI queries the current policy on the same observation stream without executing its predictions. The Energy Score compares the human action trajectory with policy inference and triggers collection when their discrepancy indicates an out-of-distribution region. In a separate feedback loop, low online advantage predictions identify essential segments for refining a progress-based advantage estimator. The updated estimator then guides advantage-conditioned behavioral cloning using a balanced mixture of base demonstrations and new policy data. This design preserves the iterative and policy-aware nature of human-in-the-loop learning while decoupling data collection from robot deployment. Experiments on four real-world tasks spanning long-horizon and precise manipulation show that HIL-UMI achieves consistent improvement over SFT and benefits from both targeted collection and advantage refinement. Moreover, HIL-UMI outperforms HG-DAgger on Clean Up Table with lower per-frame collection time, suggesting a scalable path for VLA post-training across operators and locations.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.20659 [cs.RO] |
| (or arXiv:2609.20659v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2609.20659 arXiv-issued DOI via DataCite |
Submission history
From: Zimu Han [view email]
[v1]
Thu, 17 Sep 2026 16:38:37 UTC (21,425 KB)
[v2]
Thu, 8 Oct 2026 12:57:50 UTC (21,427 KB)
来源：arXiv cs.AI · arxiv.org