arXiv cs.AI· Wo Wei Lin, Ethan Rathbun, Enrico Marchesini, Xiang Zhi Tan·· 14 小时前AI 评分29
MAVIC：面向指令合规的协作多智能体强化学习宏动作价值修正方法
Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning
AI 导读
针对指令打断宏动作时 Bellman 更新跨指令上下文耦合价值估计、导致估值不一致的问题，作者提出 MAVIC，在指令边界修正 Bellman 备份，校正新指令目标并恢复当前目标下的延续价值。
正文 · 原文
Abstract:Multi-agent reinforcement learning (MARL) in real-world use cases may need to adapt to external natural language instructions that interrupt ongoing behavior and conflict with long-horizon objectives. However, conditioning rewards on instructions introduces a fundamental failure mode as Bellman updates couple value estimates across instruction contexts, leading to inconsistent values when instructions interrupt macro-actions. We propose Macro-Action Value Correction for Instruction Compliance (MAVIC), which corrects Bellman backups at instruction boundaries by correcting the incoming instruction objective and restoring the continuation value under the current objective. Unlike reward shaping, MAVIC modifies the bootstrapping target itself, enabling consistent value estimation under stochastic instruction switching within a unified policy. We provide theoretical analysis and an actor-critic implementation, and show that MAVIC achieves high instruction compliance while preserving base task performance in increasingly complex cooperative multi-agent environments.
| Comments: | NeurIPS 2026 Atlanta Poster |
| Subjects: | Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2605.12655 [cs.AI] |
| (or arXiv:2605.12655v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.12655 arXiv-issued DOI via DataCite |
Submission history
From: Wo Wei Lin [view email]
[v1]
Tue, 12 May 2026 19:01:16 UTC (356 KB)
[v2]
Fri, 5 Jun 2026 16:06:40 UTC (2,744 KB)
[v3]
Wed, 10 Jun 2026 15:03:44 UTC (2,744 KB)
[v4]
Thu, 29 Oct 2026 15:51:53 UTC (911 KB)
来源：arXiv cs.AI · arxiv.org