arXiv cs.MA· Jie Fu, Anamika Dubey·· 9 天前AI 评分15
有限规模 MDP 智能体群体的聚合 reach-avoid 机会约束策略合成
Policy Synthesis for Finite Populations of MDP Agents under Aggregate Reach-Avoid Chance Constraints
AI 导读
针对有限规模 MDP 智能体群体在聚合 reach-avoid 机会约束下的策略合成问题，研究者提出通过离散时间 Lyapunov 递推传播经验密度的二阶矩（方差），并利用 Cantelli 不等式将机会约束转化为可处理的矩确定性条件，再嵌入基于梯度的序列凸近似过程。
正文 · 原文
Abstract:Consider a finite population of agents with decoupled Markov transition dynamics and empirical-density feedback, subject to the following constraints: with probability at least $1-\delta_r$, at least a fraction $\alpha_r$ of agents must reach a target region at some time $t^*$, while, at each time up to $t^*$, the unsafe population fraction must remain below $\beta_u$ with probability at least $1-\delta_u$. However, standard mean-field methods enforce these constraints only in expectation, which fails to account for stochastic fluctuations at finite fleet size $N$. To address this control problem, we propagate the second-order moment (variance) of the empirical density alongside the mean-field trajectory via a discrete-time Lyapunov recursion, and apply the Cantelli inequality to convert chance constraints into tractable deterministic conditions on the moments of the empirical density. We then incorporate these moment-based surrogate constraints into a gradient-based sequential convex approximation procedure for density-feedback policy synthesis. We further introduce additional moment-error bounds to construct a rigorous finite-$N$ certificate. The method is evaluated on a gridworld environment and a power-system EV-charging aggregation problem and compared with a standard deterministic population-level LP baseline.
| Comments: | 8 pages, 2 figures. Submitted to the 2027 American Control Conference |
| Subjects: | Systems and Control (eess.SY); Multiagent Systems (cs.MA) |
| MSC classes: | 93E20, 90C40, 90C15, 49N80, 93A16 |
| ACM classes: | I.2.8; G.1.6; I.2.11 |
| Cite as: | arXiv:2610.12028 [eess.SY] |
| (or arXiv:2610.12028v1 [eess.SY] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12028 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jie Fu [view email]
[v1]
Thu, 8 Oct 2026 14:23:58 UTC (111 KB)
来源：arXiv cs.MA · arxiv.org