arXiv cs.MA· Zeyuan Zhang, Yiyun Yang, Yuan Cheng, Herbert Zheng Wu, Yusi Chen·· 13 小时前AI 评分24
MA-UITD：从联合演示中离线推断多智能体奖励，刻画连续社会行为
MA-UITD: Offline Multi-Agent Reward Inference for Characterizing Continuous Social Behavior
AI 导读
研究者提出离线多智能体逆强化学习方法 MA-UITD，可从联合演示中推断各智能体的显式奖励，无需已知环境动力学或内层策略优化循环。该方法用信念条件后继特征表示单边长期价值，在离散动作的协作、竞争与一般和任务中优于基线，并在连续动作设置下推断出行为兼容的奖励。应用于社会觅食轨迹数据时，MA-UITD 为领导者与跟随者推断出不同的空间与交互相关奖励结构。
正文 · 原文
Abstract:Multi-agent inverse reinforcement learning offers a powerful framework not only for imitation learning but also for understanding the neural computations supporting social interactions and decision-making. Inferring rewards from interacting agents is challenging because observed choices reflect both personal preferences and expectations about others. Existing methods often require additional environment interaction, repeated policy optimization as an inner loop, or restrictive assumptions about environment dynamics and action spaces, limiting their applicability to fixed behavioral datasets. We introduce Multi-Agent Unilateral Inverse Temporal Difference Learning (MA-UITD), an offline method for inferring explicit agent-specific rewards from joint demonstrations in continuous-state environments with either discrete or bounded continuous actions. Its core idea is to represent the long-term consequences of each agent's own actions through belief-conditioned successor features that average over other agents' behavior. Shared reward features and agent-specific preference vectors connect these unilateral values to the target reward function, while inverse temporal-difference consistency and noise-contrastive behavioral learning enable inference without known dynamics or an inner policy-optimization loop. Across simulated cooperative, competitive, and general-sum tasks, MA-UITD outperforms baselines in discrete-action settings and infers behaviorally compatible reward profiles in continuous-action settings. When applied to experimentally collected social-foraging trajectories, MA-UITD inferred distinct spatial and interaction-dependent reward structures for leaders and followers, providing a quantitative characterization of latent behavioral objectives that may be relevant for understanding the neural computations underlying social behavior.
| Subjects: | Machine Learning (cs.LG); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.29237 [cs.LG] |
| (or arXiv:2610.29237v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29237 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zeyuan Zhang [view email]
[v1]
Wed, 21 Oct 2026 05:46:27 UTC (359 KB)
来源：arXiv cs.MA · arxiv.org