arXiv cs.AI· Jongchan Park, Hyukjun Yang, Narim Jeong, Donghwan Lee·· 10 小时前AI 评分31
AL-LBLP：用增广拉格朗日方法求解 Lifted Bellman 线性规划
An Augmented Lagrangian Approach to Lifted Bellman Linear Programming
AI 导读
研究将 Lifted Bellman Linear Programming（LBLP）的固定系数铰链惩罚改为乘子法求解，系数由对偶上升学习，仅保留一个惩罚参数 ρ，凸情形下不动点即 LBLP 的 KKT 对。
正文 · 原文
Abstract:Temporal-difference critics regress onto a bootstrapped target and usually need a target network. Lifted Bellman Linear Programming (LBLP) instead expresses the in-sample optimal values of offline RL as the solution to a linear program over the dataset. Its neural solver ALBUM relaxes the constraints into hinge penalties with fixed coefficients, and its guarantees are conditions on coefficients chosen in advance. We solve the same program with the method of multipliers: the coefficients become multipliers learned by dual ascent, with one penalty parameter $\rho$. In the convex case the fixed points are the KKT pairs of LBLP, and at every stationary point the applied constraint weights balance the objective exactly. The critic can serve a Gaussian actor, a flow actor, and best-of-N sampling, and for the Gaussian actor we propose an advantage-normalized DDPG+BC objective. With a single $Q$ and $V$ and no target network, our AL-LBLP with best-of-N reaches 60.1\% average success on the 50 state-based OGBench tasks. On long-horizon tasks, it reaches 81.5% on humanoidmaze-giant and 79.8% on antmaze-giant. It does not collapse up to batch size 8192.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.14183 [cs.AI] |
| (or arXiv:2611.14183v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14183 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jongchan Park [view email]
[v1]
Tue, 10 Nov 2026 15:05:22 UTC (132 KB)
来源：arXiv cs.AI · arxiv.org