arXiv cs.AI· Kushal Kedia, Tyler Ga Wei Lum, Jeannette Bohg, C. Karen Liu·· 10 天前AI 评分33
SimToolReal：面向零样本灵巧工具操作的以物体为中心策略
SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation
AI 导读
SimToolReal 通过程序化生成大量工具类物体基元并训练单一 RL 策略，实现测试时无需任何物体或任务特定训练的通用灵巧工具操作。该策略比此前的重定向和固定抓取方法性能高 37%，并达到针对特定目标物体和任务训练的专家 RL 策略水平。在 24 项任务、12 个物体实例、6 类工具的 120 次真实世界 rollout 中，SimToolReal 展现出强零样本泛化能力。
正文 · 原文
Abstract:The ability to manipulate tools significantly expands the set of tasks a robot can perform. Yet, tool manipulation represents a challenging class of dexterity, requiring grasping thin objects, in-hand object rotations, and forceful interactions. Since collecting teleoperation data for these behaviors is challenging, sim-to-real reinforcement learning (RL) is a promising alternative. However, prior approaches typically require substantial engineering effort to model objects and tune reward functions for each task. In this work, we propose SimToolReal, taking a step towards generalizing sim-to-real RL policies for tool manipulation. Instead of focusing on a single object and task, we procedurally generate a large variety of tool-like object primitives in simulation and train a single RL policy with the universal goal of manipulating each object to random goal poses. This approach enables SimToolReal to perform general dexterous tool manipulation at test-time without any object or task-specific training. We demonstrate that SimToolReal outperforms prior retargeting and fixed-grasp methods by 37% while matching the performance of specialist RL policies trained on specific target objects and tasks. Finally, we show that SimToolReal generalizes across a diverse set of everyday tools, achieving strong zero-shot performance over 120 real-world rollouts spanning 24 tasks, 12 object instances, and 6 tool categories.
| Comments: | 23 pages, 16 figures, 3 tables. Project page: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.16863 [cs.RO] |
| (or arXiv:2602.16863v3 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2602.16863 arXiv-issued DOI via DataCite |
Submission history
From: Tyler Lum [view email]
[v1]
Wed, 18 Feb 2026 20:42:39 UTC (44,628 KB)
[v2]
Tue, 24 Feb 2026 17:10:02 UTC (44,629 KB)
[v3]
Tue, 6 Oct 2026 05:50:23 UTC (44,629 KB)
来源：arXiv cs.AI · arxiv.org