arXiv cs.AI· Kevin Avdiu, Seongjin Bien, Yannik Blei, Philine Meister, Johannes Hechtl, Philipp Schmitt, Kai Wurm, Georg von Wichert, Wolfram Burgard·· 1 天前AI 评分31
RUMI：匹配 Robotiq 2F-85 的通用操作接口，结合 AR 与触觉反馈用于机器人学习
RUMI: A Robotiq-Matched Universal Manipulation Interface for Robot Learning with Augmented Reality and Haptic Feedback
AI 导读
RUMI 是一款与 Robotiq 2F-85 夹爪匹配的手持通用操作接口，配备可更换指尖传感器，在保持目标夹爪几何结构的同时支持接触前自然输入与接触后力测量。研究将追踪系统集成进 VR 头显，实时提供运动学可行性的 AR 反馈，并引入支持力感知遥操作的触觉反馈机制。用户研究中，启用实时机器人可视化与位姿初始化后，出现关节限位违规的回合为 3/50，未启用 AR 时为 16/50。
正文 · 原文
Abstract:Many manipulation tasks require carefully controlled contact forces for successful execution, yet large-scale, force-aware datasets remain scarce. Handheld interfaces can help by transmitting interaction forces directly through the operator's arm, but most existing designs are tailored to simple linear grippers and do not match the nonlinear mechanics of widely used robotic grippers. We present RUMI, a handheld universal manipulation interface matched to the Robotiq 2F-85 with exchangeable fingertip sensors that preserves the target gripper geometry while enabling natural input before contact and force measurement after contact. By integrating the tracking system into a virtual reality headset, our method provides real-time augmented reality feedback about the kinematic feasibility of motion on the target robot platform. We further introduce a haptic feedback mechanism that supports force-aware teleoperation for simulated and real robots. A user study shows fewer episodes containing joint-limit violations with live robot visualization and pose initialization (3/50) than without augmented reality (16/50). We additionally train imitation policies from demonstrations, collected using two different fingertip modalities enabled by the gripper-matched hardware, and assess the haptic interface's capability for collecting force-aware teleoperation datasets in simulation.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.15802 [cs.RO] |
| (or arXiv:2611.15802v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15802 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yannik Blei [view email]
[v1]
Wed, 11 Nov 2026 18:17:42 UTC (4,299 KB)
来源：arXiv cs.AI · arxiv.org