arXiv cs.CL· Buzhao Liu, Jianing Ye, Xiu Yuan, Yangtong Feng, Chengsong Huang, Tyler Dinh, Yevgeniy Vorobeychik, Ning Zhang, Chongjie Zhang·· 13 小时前AI 评分42
RoboAuto：通过多层级物理理解实现自主机器人程序搜索
RoboAuto: Autonomous Robot Program Search through Multi-Level Physical Understanding
AI 导读
RoboAuto 是一个面向机器人程序搜索的框架，通过分层知识库、场景接地和多阶段规划三个组件，在对象、场景与交互三个层面显式组织物理理解。它在 LIBERO-PRO 短程任务上达到 89.1% 成功率，在 LIBERO-PRO Long 和 LIBERO-Plus 上的平均成功率超过 ASPIRE 两倍，并在三个长程真实机器人任务上优于 ASPIRE 和 π_{0.5}。
正文 · 原文
Abstract:Recently, LLM-based autonomous agents have shown strong capabilities in iteratively generating, evaluating, and refining executable solutions. In robotic manipulation, Code-as-Policy lets these agents search over modular and editable robot programs. However, effective program search requires physical understanding: agents must understand how objects can be manipulated, how scene geometry constrains actions, and why interactions succeed or fail. Existing methods rely mainly on execution feedback for program refinement, often leaving these physical questions unresolved. We introduce RoboAuto, a framework for robot program search that explicitly organizes physical understanding at the object, scene, and interaction levels through three components. First, a hierarchical knowledge base combines executable skills with object-centric knowledge of affordances, spatial relations, and action effects, learned from embodied data and online task solving. Second, scene grounding connects this knowledge to the current environment by identifying relevant objects and resolving action-critical quantities. Third, multi-stage planning with self-designed experiments checks whether each stage produces its expected physical effects and actively tests competing hypotheses to resolve physical uncertainties. RoboAuto achieves superior performance across five benchmark families, attaining an 89.1% success rate on short-horizon LIBERO-PRO tasks and more than twice the mean success rate of ASPIRE on LIBERO-PRO Long and LIBERO-Plus. It also outperforms ASPIRE and $\pi_{0.5}$ on three challenging long-horizon real-robot tasks requiring precise manipulation. We further show that the knowledge base transfers across simulators and embodiments, and that program search performance scales with its size. Project page: this https URL
| Comments: | Project page: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.36533 [cs.RO] |
| (or arXiv:2610.36533v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36533 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Buzhao Liu [view email]
[v1]
Tue, 27 Oct 2026 02:41:22 UTC (9,278 KB)
来源：arXiv cs.CL · arxiv.org