arXiv cs.AI· Nick Leenders, Roy Lindelauf, Joost van Oijen, Boris Cule·· 9 天前AI 评分18
基于约束命令条件强化学习与 Bandit 策略选择的 MicroRTS 智能体
Constrained Command-Conditioned Reinforcement Learning with Bandit Strategy Selection in Real-Time Strategy Games
AI 导读
研究者提出一种约束命令条件 PPO 执行器，配合 Thompson 采样 Bandit 策略器，在 MicroRTS 中按游戏内观测估计对手策略并选择命令元组，而非依赖对手身份。
正文 · 原文
Abstract:Deep reinforcement learning agents reach strong performance in real-time strategy games but can be brittle against opponents outside their training distribution. Separating strategic command selection from learned unit control allows different strategies to be selected for different opponents while reusing the same execution policy. This requires an executor that can follow different commands and measurable criteria for assessing whether it does so. We introduce a constrained command-conditioned Proximal Policy Optimization (PPO) policy, the executor, for MicroRTS, a real-time strategy environment. Discrete commands specify strategic objectives and behavioral requirements for economy, army composition, military posture, and worker policy over multiple environment steps; the executor determines the unit-level actions used to fulfill them. A Thompson-sampling bandit acts as the strategist, selecting command tuples from an estimate of the opponent's strategy built from in-game observations rather than opponent identity. In a controlled comparison with a flat PPO baseline trained with the same architecture, budget, curriculum and self-play league, the strategist-executor system wins significantly more often against three of the four strongest opponents on a training map, including the two strongest held-out ones (0.55 to 0.97 and 0.01 to 0.34), with no significant difference against the others.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.11663 [cs.AI] |
| (or arXiv:2610.11663v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11663 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: NIck Leenders [view email]
[v1]
Thu, 8 Oct 2026 10:37:33 UTC (178 KB)
来源：arXiv cs.AI · arxiv.org