arXiv cs.AI· Erwan C. Escudie, Matthia Sabatelli, Jilles Steeve Dibangoye·· 13 小时前AI 评分26
面向零和 POSG 的保守策略迭代算法
Conservative Policy Iteration For Zero-Sum POSGs
AI 导读
研究者提出一种面向有限时域零和部分可观测随机博弈（zs-POSG）的近似策略迭代算法，为每名玩家维护一个策略并沿保守策略更新诱导的占据轨迹联合更新，每次更新只用当前联合策略诱导的单个支撑超平面，而非逼近完整中心价值函数。在有限时域零和 POSG 基准上，该方法在小时间跨度下具竞争力，较大跨度下相比 PBVI、HSVI 和 CFR+ 常能改善运行时与可利用度的权衡。
正文 · 原文
Abstract:Occupancy-state reductions make dynamic programming possible for zero-sum partially observable stochastic games (zs-POSGs). Existing point-based solvers usually optimise a central value function. This restores Bellman recursion, but it requires point-based updates over high-dimensional occupancy spaces. We introduce an approximate policy-iteration algorithm for finite-horizon zs-POSGs. The method maintains one policy per player and updates both jointly along occupancy trajectories induced by conservative policy updates. Instead of approximating a full central value function, each update uses a single supporting hyperplane induced by the current joint policy. The algorithm alternates local policy improvements, conservative updates, and persistent exploration. Computation is therefore concentrated on occupancy regions where the current joint policy is exploitable. On finite-horizon zero-sum POSG benchmarks, the method is competitive at small horizons and often improves the runtime-exploitability trade-off at larger horizons compared with PBVI, HSVI, and CFR+.
| Subjects: | Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT) |
| Cite as: | arXiv:2610.36847 [cs.AI] |
| (or arXiv:2610.36847v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36847 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Erwan Escudie [view email]
[v1]
Tue, 27 Oct 2026 08:45:33 UTC (321 KB)
来源：arXiv cs.AI · arxiv.org