arXiv cs.CL· Mengzhao Jia, Yang Lin, Xixin Zhang, Zhihan Zhang, Meng Jiang·· 10 小时前AI 评分52
Agent as Policy:通用编码智能体直接驱动机器人操作
Agent as Policy for Robotic Manipulation
AI 导读
论文提出 Agent as Policy(AGP),让通用编码智能体无需任务或环境特定训练即可直接驱动机器人,由智能体解读视觉信息、编写可执行程序、下发运动指令并根据物理结果修正动作。
正文 · 原文
Abstract:We demonstrate that a general-purpose coding agent can directly drive a physical robot throughout task execution without any task-specific or environment-specific training. We introduce Agent as Policy (AGP), which places task planning and execution under the agent's control. Given a task and a robot interface, the agent interprets visual evidence, writes executable programs, issues motion commands, and revises its actions in response to physical outcomes. This brings the agent's reasoning and programming capabilities into continuous interaction with the physical world. We study AGP across multiple real-world manipulation tasks spanning precision manipulation, dynamic motions, and deformable objects. These include assembly from human videos, block construction from goal images, dice flipping, targeted throwing, and bimanual towel folding. AGP achieves success rates of at least 80% in seven of eight task configurations and significantly outperforms previous agentic robot systems. We further study efficiency through task experience accumulation and find that reusing saved procedures and programs shortens execution time across repeated trials. These findings support a path for general-purpose agents to act as robot policies, extending their autonomy to physical manipulation through runtime reasoning, programming, and interaction.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.12541 [cs.CL] |
| (or arXiv:2609.12541v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.12541 arXiv-issued DOI via DataCite |
Submission history
From: Mengzhao Jia [view email]
[v1]
Fri, 11 Sep 2026 07:51:39 UTC (36,102 KB)
[v2]
Mon, 14 Sep 2026 08:11:02 UTC (19,719 KB)
[v3]
Mon, 28 Sep 2026 22:20:30 UTC (18,417 KB)
[v4]
Wed, 18 Nov 2026 23:18:56 UTC (23,264 KB)
来源:arXiv cs.CL · arxiv.org