arXiv cs.MA· Leyang Shen, Yaxin Gong, Yang Zhang, Chenxiao Fan, Jiakai Wang, Dong Wang, Yang Liu, Fuli Feng, Chun Kai Ling, Tat-Seng Chua·· 2 天前AI 评分29
训练 LLM 智能体与多智能体系统的纠偏能力
Learning to Steer LLM Agents and Multi-Agent Systems
AI 导读
针对 LLM 智能体与多智能体系统执行中偏离总体目标的"目标漂移"问题，研究者提出一个在旁监督执行的 steering agent，通过低成本快路径判断何时干预、高成本慢路径决定如何干预。两条路径用强化学习联合训练，结合成对排序目标与 GRPO。在信息检索类搜索任务上，该方法对单智能体和多智能体系统均提升了性能与效率，且能泛化到域外任务、迁移到未见过的执行器，并支持无需重训的可控干预频率。
正文 · 原文
Abstract:Large language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these systems are prone to deviating from the overall goal as execution unfolds, which wastes execution resources and leads to undesired outcomes, a failure termed goal drift. In this work, we propose a steering agent that supervises execution on the fly and steers the system back on track when drift occurs. The steering agent observes the execution context, deciding when to intervene with a low-cost fast path and how to intervene with a high-cost slow path. To unlock its steering intelligence, the ability to assess goal drift and generate effective interventions, we jointly train the two paths with reinforcement learning, combining a pairwise ranking objective with group-relative policy optimization (GRPO). Experiments on information-seeking search tasks show that the steering agent provides effective supervision for both single-agent and multi-agent systems, improving their performance and efficiency. The learned steering intelligence generalizes to out-of-domain tasks, transfers to unseen executors, and supports controllable intervention frequency without retraining.
| Comments: | 21 pages, 6 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.22316 [cs.LG] |
| (or arXiv:2610.22316v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22316 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Leyang Shen [view email]
[v1]
Thu, 15 Oct 2026 17:51:17 UTC (240 KB)
来源：arXiv cs.MA · arxiv.org