arXiv cs.AI· Nikola Milosevic, Nicol\'as Hinrichs, Nico Scherf·· 3 天前AI 评分26
主动推理可建模为凸马尔可夫决策过程
Active Inference as a Convex Markov Decision Process
AI 导读
研究将主动推理（AIF）的期望自由能（EFE）最小化重构为策略优化问题，证明闭环控制策略下的 EFE 最小化可表述为凸马尔可夫决策过程（MDP）。该视角揭示策略相关的奖励预测误差是沿时间反向传播 EFE 的自然梯度，而非沿层级向上传播。将世界模型学习与策略优化耦合后，主动推理呈现表演式强化学习（performative RL）的结构。
正文 · 原文
Abstract:Active Inference (AIF) frames adaptive behavior as the minimization of expected free energy (EFE), combining epistemic and pragmatic objectives within a single variational principle. We frame AIF as policy optimization and show that, for closed-loop control policies, EFE minimization can be formulated as a convex Markov decision process (MDP). This perspective reveals that policy-dependent reward prediction errors transmit natural gradients of the expected free energy backwards in time rather than up a hierarchy. Finally, we show that coupling world-model learning with policy optimization gives active inference the structure of performative reinforcement learning. Together this places EFE minimization within modern reinforcement learning and optimization theory and opens a route toward principled algorithms for active inference.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2607.20152 [cs.LG] |
| (or arXiv:2607.20152v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.20152 arXiv-issued DOI via DataCite |
Submission history
From: Nikola Milosevic [view email]
[v1]
Wed, 22 Jul 2026 13:52:40 UTC (308 KB)
[v2]
Mon, 21 Sep 2026 10:39:39 UTC (314 KB)
[v3]
Tue, 13 Oct 2026 13:55:26 UTC (309 KB)
来源：arXiv cs.AI · arxiv.org