arXiv cs.AI· Hieu Tran, Hong Yu·· 13 小时前AI 评分31
ArcFlow：为多种求解器预算训练流策略
ArcFlow: Training Flow Policies for Multiple Solver Budgets
AI 导读
ArcFlow 是一种面向部署的流匹配策略训练算法，让单一策略在指定范围的多个求解器预算下都能工作。在 DM Control、操作与运动任务中，ArcFlow 在 30 个任务里的 25 个上取得五种流策略基线中最高的平均单步回报。方法结合动作对齐监督、预算暴露与最差预算检查点选择，且不改变模型架构与部署成本。
正文 · 原文
Abstract:Flow-matching policies adjust inference cost through the number of solver steps, but reinforcement learning at a single fixed solver budget can hide degradation at cheaper deployment budgets and even erase useful one-step behavior present in a pretrained policy. We introduce Action-Ratio-Clip Flow (ArcFlow), a deployment-aware algorithm for training one flow policy across a specified range of solver budgets. ArcFlow combines action-aligned supervision, budget exposure, and worst-budget checkpoint selection. For reconstruction objectives, its bridge aggregates temporal probes into one surrogate ratio and one PPO-style clip per action, with a boundary residual exactly equal to deployed one-step reconstruction error. Cycling exposes the shared field to the deployment range, and selection optimizes its performance floor. For native velocity objectives, multi-budget endpoint supervision applies the same principle without replacing the native update. Both realizations preserve the architecture and deployment-time cost. Across DM Control, manipulation, and locomotion, ArcFlow attains the highest mean one-step return among five flow-policy baselines on 25 of 30 tasks under each method's reported training settings. Component studies identify complementary improvements from boundary supervision, the fixed-grid bridge, budget exposure, and deployment selection. These results establish deployment-budget-aware training as an effective route to strong low-step control with one policy serving multiple inference costs.
| Comments: | 19 pages, 4 figures, 4 tables |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.28788 [cs.LG] |
| (or arXiv:2610.28788v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28788 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hieu Tran [view email]
[v1]
Tue, 20 Oct 2026 18:18:42 UTC (1,065 KB)
来源：arXiv cs.AI · arxiv.org