arXiv cs.AI· Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh, Xiaoyu Huang, Takara E. Truong, Carmelo Sferrazza, Yi Ma, Rocky Duan, Pieter Abbeel, Guanya Shi, Karen Liu, Angjoo Kanazawa·· 3 天前AI 评分32
Flow Policy Gradients 如何用于机器人控制训练
Flow Policy Gradients for Robot Control
AI 导读
研究提出改进的 flow matching 策略梯度目标，绕过似然计算，使机器人控制策略可训练为更 expressive 的分布。该方法在腿足运动、人形动作跟踪和操作任务上均取得成功，并在两台人形机器人上实现稳健的 sim-to-real 迁移。消融分析显示，策略可从零训练时利用 flow 表示进行探索，微调鲁棒性也优于基线。
正文 · 原文
Authors:Brent Yi, Hongsuk Choi, Himanshu Gaurav Singh, Xiaoyu Huang, Takara E. Truong, Carmelo Sferrazza, Yi Ma, Rocky Duan, Pieter Abbeel, Guanya Shi, Karen Liu, Angjoo Kanazawa
Abstract:Likelihood-based policy gradient methods are the dominant approach for training robot control policies from rewards. These methods rely on differentiable action likelihoods, which constrain policy outputs to simple distributions like Gaussians. In this work, we show how flow matching policy gradients, a recent framework that bypasses likelihood computation, can be made effective for training and finetuning more expressive policies in challenging robot control settings. We introduce an improved objective that enables success in legged locomotion, humanoid motion tracking, and manipulation tasks, as well as robust sim-to-real transfer on two humanoid robots. We then present ablations and analysis on training dynamics. Results show how policies can exploit the flow representation for exploration when training from scratch, as well as improved finetuning robustness over baselines.
| Comments: | Project webpage: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.02481 [cs.RO] |
| (or arXiv:2602.02481v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2602.02481 arXiv-issued DOI via DataCite |
Submission history
From: Hongsuk Choi [view email]
[v1]
Mon, 2 Feb 2026 18:56:49 UTC (10,944 KB)
[v2]
Tue, 13 Oct 2026 08:31:31 UTC (16,799 KB)
来源：arXiv cs.AI · arxiv.org