arXiv cs.MA· Alireza Saleh Abadi, Leen-Kiat Soh, Adam Eck, Prashant Doshi·· 11 小时前AI 评分26
TANDEM：用集中式与去中心化双读评论家弥合多智能体训练与执行的信息差
TANDEM: Balancing Centralized and Decentralized Critics to Address the Information Gap in Complex Multi-Agent Environments
AI 导读
TANDEM 在每步训练中保留一个集中式评论家并做两种读取：基于全局状态的集中式读取，以及仅限智能体执行时可见信息的去中心化读取，再用无参数门控在两者一致处按去中心化读取训练策略，仅在局部视角误导时恢复集中式读取。
正文
Abstract:Cooperative multi-agent reinforcement learning under centralized training with decentralized execution (CTDE) shapes decentralized policies with a centralized critic that observes the global state, then removes the critic at execution. The policies are optimized to rely on information they will not have when they act, an information gap between training and execution. It widens under partial observability, observation noise, and openness, where the set of agents and tasks present changes during an episode. Existing methods transfer centralized guidance in one direction only, removing or distilling the critic once, so the policies never rehearse acting without the critic while they can still be corrected. We propose TANDEM, which keeps one centralized critic and reads it two ways at each training step: a centralized read from the global state, and a decentralized read restricted to the information each agent will have at execution. A parameter-free gate compares the two reads for each agent and trains the policy on the decentralized read where they agree, reinstating the centralized read only where the local view is misleading. A shared target pulls the decentralized read toward the centralized one. We formalize the setting as a task- and agent-open decentralized partially observable Markov decision process (TAO-Dec-POMDP) and evaluate TANDEM on three cooperative domains, each isolating one form of the information gap: discovery for partial observability, sampling for observation noise, and openness in wildfire suppression. We compare against six CTDE baselines and ablate the gate and the shared target; the supplementary material holds the full variant sweep, diagnostics, and learning curves. We measure return at execution, once the centralized critic is removed. TANDEM is the best method on eight of the nine setups and never significantly worse than any baseline.
| Subjects: | Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.37592 [cs.MA] |
| (or arXiv:2610.37592v1 [cs.MA] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37592 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Alireza Saleh Abadi [view email]
[v1]
Tue, 27 Oct 2026 17:28:43 UTC (1,994 KB)
来源：arXiv cs.MA · arxiv.org