arXiv cs.AI· Andrew McAlinden, Caroline Wang, Steven Shi, Arrasy Rahman, Elad Liebman, Peter Stone·· 12 小时前AI 评分20
集中式评论家如何服务 N-agent 临时团队协作
Centralized Critics for N-Agent Ad Hoc Teamwork
AI 导读
研究比较了 N-agent 临时团队协作（NAHT）中两类优势估计器：仅依赖己方智能体历史的受控历史估计器，以及额外纳入不可控队友历史的联合历史估计器。作者证明两者均为无偏估计，并刻画了联合历史估计方差严格更低的成立条件；在多项 NAHT 任务上，联合历史评论家达到或超过受控历史评论家的表现，样本效率与最终回报均更优。
正文
Abstract:Centralized training with decentralized execution (CTDE) has become the dominant paradigm in multi-agent reinforcement learning (MARL), motivating extensive study of what privileged information should be made available to the critic during training. While prior work has examined the role of global state in fully cooperative settings, the design of centralized critics for N-agent ad hoc teamwork - in which dynamically varying set of learning agents must coordinate with an arbitrary set of uncontrolled teammates drawn from an unknown distribution - remains poorly understood. We study this question by comparing two natural classes of advantage estimators: controlled history estimators, which condition only on the observation and action histories of the learner's own agents, and joint history estimators, which additionally condition on the histories of the uncontrolled teammates. We prove that both estimators are unbiased and characterize the conditions under which the joint history estimate attains strictly lower variance than its controlled counterpart. Empirically, joint history critics match or outperform controlled history critics across a range of NAHT tasks, yielding improved sample efficiency and final returns. Together, these results provide both a theoretical justification and an empirical recommendation for incorporating uncontrolled agent histories into centralized critics for N-agent ad hoc teamwork.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.34799 [cs.AI] |
| (or arXiv:2610.34799v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34799 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Andrew McAlinden [view email]
[v1]
Mon, 26 Oct 2026 03:41:48 UTC (2,681 KB)
来源：arXiv cs.AI · arxiv.org