arXiv cs.MA· Lucia Pezzetti, Nicolas Lanzetti, Antonio Terpin, Florian D\"orfler, Giorgia Ramponi·· 11 小时前AI 评分38
合作多智能体强化学习的分离原理：SALT 算法
A Separation Principle for Cooperative Multi-Agent Reinforcement Learning
AI 导读
研究者提出合作多智能体强化学习（MARL）的分离原理：群体 cost-to-go 函数可由最优传输问题给出上界，其传输代价为目标条件化的单智能体 cost-to-go 函数，从而把多智能体问题简化为学习单智能体策略并用最优传输协调智能体。
正文 · 原文
Abstract:We study cooperative multi-agent reinforcement learning (MARL) problems in which agents evolve under decoupled noisy dynamics but share a population objective. A canonical example is a mobility operator that orchestrates a fleet of vehicles to meet customer demand. The stochastic and time-varying nature of these problems calls for learning approaches, yet existing MARL algorithms do not scale: learning assignment and routing jointly over the full state-action space becomes prohibitive as the number of agents grows. Formulating the problem over the space of probability measures, we prove a separation principle: the population cost-to-go function is upper bounded by an optimal transport problem whose transportation cost is the target-conditioned single-agent cost-to-go function. This simplifies the multi-agent problem to learning a single-agent policy and coordinating the agents via optimal transport. Building on this principle, we introduce SALT (Separation-based Assignment and Learning via optimal Transport), a family of algorithms that learn a target-conditioned single-agent policy using any standard reinforcement learning algorithm, and then coordinate the agents via optimal transport. Since learning occurs at the single-agent level, SALT scales to fleet sizes where MARL baselines become intractable and transfers without retraining to fleet sizes unseen during training. In a vehicle-delivery experiment on the southern Manhattan road network, where MARL baselines are computationally prohibitive, SALT improves travel time by 15-21% over a shortest-path routing heuristic. In stochastic grid worlds, SALT reduces average travel times by 23-61% relative to MARL baselines and scales to population sizes where these methods fail.
| Subjects: | Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2611.09644 [cs.MA] |
| (or arXiv:2611.09644v1 [cs.MA] for this version) | |
| https://doi.org/10.48550/arXiv.2611.09644 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lucia Pezzetti [view email]
[v1]
Fri, 6 Nov 2026 15:57:18 UTC (221 KB)
来源：arXiv cs.MA · arxiv.org