arXiv cs.AI· Ajay Vikram Periasami, Anjaneya Gupta, Ethan Cheng, Ivan Rosales·· 14 小时前AI 评分26
面向 Balatro 游戏的强化学习：快速模拟器与双策略训练方法
Reinforcement Learning for the Game of Balatro: A Fast Simulator and Two-Policy Training Approach
AI 导读
研究团队为卡牌游戏 Balatro 构建端到端强化学习流水线，包含经 BalatroBot 与线上客户端自动状态比对的 Rust 高速模拟器、带动态合法动作掩码的压缩观测-动作接口，以及将出牌与商店决策拆分的多智能体 PPO 架构。
正文 · 原文
Abstract:Balatro, a 2024 roguelike deckbuilder, presents three difficulties that have historically derailed reinforcement learning methods: a 3,000-action state-dependent action space, horizons of several hundred decisions with sparse rewards, and pervasive stochasticity in both card draws and shop generation. Published RL results on the game remain scarce. We introduce an end-to-end pipeline comprising (i) a high-throughput Rust simulator that is automatically state-diffed against the live game client via BalatroBot, (ii) a condensed observation-action interface with dynamic per-step masking of legal moves, and (iii) a multi-agent PPO architecture that decomposes play into a tactical blind policy and a strategic shop policy that share the full game state but act in disjoint phases. Long-horizon credit assignment is partially addressed by per-agent reward routing with explicit cross-agent credit. On the main joint run, training metrics show a sharp phase transition in which the first-blind clear rate converges above 99% and mean episode return increases by roughly an order of magnitude, plateauing at a mean maximum ante of 3 out of 8; single-agent baselines under identical rewards oscillated between greedy and hoarding failure modes and did not stably clear the first blind. These results establish the two-policy decomposition as a strong reactive baseline for Balatro and localise the remaining gap to long-horizon strategic planning rather than tactical blind play.
| Comments: | Accepted at EXAG 2026 (AIIDE Workshop on Experimental AI in Games) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39049 [cs.LG] |
| (or arXiv:2610.39049v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39049 arXiv-issued DOI via DataCite |
Submission history
From: Ajay Vikram Periasami [view email]
[v1]
Wed, 28 Oct 2026 23:24:53 UTC (2,347 KB)
来源：arXiv cs.AI · arxiv.org