arXiv cs.AI· Zeyuan Wang, Da Li, Yulin Chen, Yuehu Gong, Yanming Guo, Liang Bai, Tianyuan Yu, Yanwei Fu·· 6 天前AI 评分22
Stochastic MeanFlow Policies：一步生成式控制的熵镜像下降方法
Stochastic MeanFlow Policies: One-Step Generative Control with Entropic Mirror Descent
AI 导读
研究者提出 Stochastic MeanFlow Policies（SMFP），一种一步生成式策略类，将 MeanFlow 噪声到动作映射与高斯重参数化结合，用可学习的条件高斯尺度提供解析熵下界，实现熵正则化镜像下降学习。在七个 MuJoCo 基准上，SMFP 以单步推理取得优于高斯与生成式基线的表现，并在稀疏奖励导航中展现出多模态动作建模与有效探索能力。
正文 · 原文
Abstract:Combining entropy regularisation with mirror descent in online off-policy reinforcement learning requires expressive policies with efficient sampling and explicit entropy control. Gaussian policies struggle to represent multimodal targets, whereas generative policies often require iterative sampling or lack tractable entropy estimates. We introduce Stochastic MeanFlow Policies (SMFP), a one-step generative policy class that combines MeanFlow-based noise-to-action mappings with Gaussian reparameterisation. The latent mapping captures diverse action modes, while a learnable conditional Gaussian scale provides an analytic entropy lower bound. We use this bound to regulate stochasticity alongside direct value guidance and advantage-weighted MeanFlow regression, enabling entropy-regularised mirror-descent learning with one-step policies. Across seven MuJoCo benchmarks, SMFP achieves strong performance against Gaussian and generative baselines with single-step inference. Additional experiments demonstrate multimodal action modelling and effective exploration in sparse-reward navigation.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.21282 [cs.LG] |
| (or arXiv:2605.21282v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.21282 arXiv-issued DOI via DataCite |
Submission history
From: Zeyuan Wang [view email]
[v1]
Wed, 20 May 2026 15:14:14 UTC (638 KB)
[v2]
Thu, 21 May 2026 06:09:02 UTC (638 KB)
[v3]
Fri, 9 Oct 2026 02:37:42 UTC (716 KB)
来源：arXiv cs.AI · arxiv.org