arXiv cs.MA· Bhavini Jeloka, Yue Guan, Panagiotis Tsiotras·· 13 小时前AI 评分29
MF-MAPPO：用平均场交互学习大规模竞争团队行为
Learning Large-Scale Competitive Team Behaviors with Mean-Field Interactions
AI 导读
研究者提出 MF-MAPPO，将 PPO 扩展到零和团队博弈的平均场框架，兼顾队内合作与队间竞争，每队仅用共享 actor 和极简信息 critic，可直接在有限种群模拟器上训练并部署到数千智能体的场景。
正文 · 原文
Abstract:While multi-agent reinforcement learning (MARL) has been proven effective across both collaborative and competitive tasks, existing algorithms often struggle to scale to large populations of agents. Recent advancements in mean-field (MF) theory provide scalable solutions by approximating population interactions as a continuum, yet most existing frameworks focus exclusively on either fully cooperative or purely competitive settings. To bridge this gap, we introduce MF-MAPPO, a mean-field extension of PPO designed for zero-sum team games that integrate intra-team cooperation with inter-team competition. MF-MAPPO employs a shared actor and a minimally informed critic per team and is trained directly on finite-population simulators, thereby enabling deployment to realistic scenarios with thousands of agents. We further show that MF-MAPPO naturally extends to partially observable settings through a simple gradient-regularized training scheme. Our evaluation utilizes large-scale benchmark scenarios using our own testing simulation platform for MF team games (MFEnv), including offense-defense battlefield tasks as well as variants of population-based rock-paper-scissors games that admit analytical solutions, for benchmarking. Across these benchmarks, MF-MAPPO outperforms existing methods and exhibits complex, heterogeneous behaviors, demonstrating the effectiveness of combining mean-field theory and MARL techniques at scale.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2504.21164 [cs.MA] |
| (or arXiv:2504.21164v4 [cs.MA] for this version) | |
| https://doi.org/10.48550/arXiv.2504.21164 arXiv-issued DOI via DataCite |
Submission history
From: Bhavini Jeloka [view email]
[v1]
Tue, 29 Apr 2025 20:31:59 UTC (13,456 KB)
[v2]
Sun, 28 Sep 2025 15:55:43 UTC (6,703 KB)
[v3]
Fri, 13 Feb 2026 11:51:28 UTC (27,181 KB)
[v4]
Wed, 21 Oct 2026 17:35:53 UTC (15,025 KB)
来源：arXiv cs.MA · arxiv.org