arXiv cs.AI· Ammar N. Abbas, Aayush Jain, Prashanth Viswanath, Nivesh Gadipudi, Giulia Vilone, Jim Gibbons·· 1 天前AI 评分34
从遥操作到长程灵巧性：评估灵巧操作的视觉运动技能与离线代理指标
From Teleoperation to Long-Horizon Dexterity: Evaluating Visuomotor Skills and Offline Proxies for Dexterous Manipulation
AI 导读
研究在五指机械手上用约 1,700 次真实机器人试验评估了覆盖七个策略族的 281 个模仿学习策略，任务为 RJ45 连接器抓取与插入。受控消融显示观测与动作表征的影响超过优化器和采样选择：点云策略中感兴趣区域裁剪、机器人基座坐标系表征与点编码器内归一化各提升成功率 70-100 个百分点，视觉-语言-动作策略改用绝对动作预测使插入成功率最高提升 70 个百分点。
正文 · 原文
Abstract:Imitation learning policies are intended to make robotic manipulation robust to variations in object pose, appearance, and environment that challenge pre-programmed motion. However, real-world performance remains difficult to predict, as conventional optimization signals such as validation loss show no consistent relationship with robot success. To identify the design choices that govern real-world performance, 281 imitation-learning policies spanning seven policy families were evaluated in approximately 1,700 real-robot trials of RJ45 connector grasping and insertion using a five-fingered robotic hand. Controlled ablations showed that observation and action representations dominated conventional optimizer and sampling choices. For point-cloud policies, region-of-interest cropping, representation in the robot-base frame, and normalization within the point encoder each improved success by 70-100 percentage points. For the vision-language-action policy, absolute rather than relative action prediction improved insertion success by up to 70 percentage points. Skill decomposition, multi-view perception, and recovery demonstrations were associated with the strongest-performing systems, although their effects were not independently isolated. Of 67 offline metrics evaluated, only five measures of predicted-motion smoothness and directness reliably correlated with insertion success (r = -0.70 to -0.80), and only within a single policy family. These results identify representation choices as key determinants of real-world imitation-learning performance and show that offline metrics can shortlist policies, but cannot replace real-robot evaluation.
| Comments: | Accepted at CoRL 2026 Workshop |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Systems and Control (eess.SY) |
| Cite as: | arXiv:2610.40828 [cs.RO] |
| (or arXiv:2610.40828v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40828 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ammar Abbas PhD [view email]
[v1]
Thu, 29 Oct 2026 18:07:37 UTC (1,397 KB)
来源：arXiv cs.AI · arxiv.org