跳到正文
arXiv cs.CY· Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Zhe Wang, Zhenhailong Wang, Cheng Qian, Xiangru Tang, Heng Ji, Jiaxuan You·· 23 小时前AI 评分38

MultiAgentBench:评估 LLM 智能体的协作与竞争

MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents

AI 导读

MultiAgentBench 是一个评估 LLM 多智能体系统协作与竞争的综合性基准,采用基于里程碑的关键绩效指标衡量任务完成度与协作质量。评测覆盖 star、chain、tree、graph 等协调拓扑及群体讨论、认知规划等策略,gpt-4o-mini 取得平均最高任务分数,graph 结构在研究场景中表现最佳,认知规划将里程碑达成率提升 3%。代码与数据集已公开。

正文 · 原文

Authors:Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Zhe Wang, Zhenhailong Wang, Cheng Qian, Xiangru Tang, Heng Ji, Jiaxuan You

View PDF HTML (experimental)

Abstract:Large Language Models (LLMs) have shown remarkable capabilities as autonomous agents, yet existing benchmarks either focus on single-agent tasks or are confined to narrow domains, failing to capture the dynamics of multi-agent coordination and competition. In this paper, we introduce MultiAgentBench, a comprehensive benchmark designed to evaluate LLM-based multi-agent systems across diverse, interactive scenarios. Our framework measures not only task completion but also the quality of collaboration and competition using novel, milestone-based key performance indicators. Moreover, we evaluate various coordination protocols (including star, chain, tree, and graph topologies) and innovative strategies such as group discussion and cognitive planning. Notably, gpt-4o-mini reaches the average highest task score, graph structure performs the best among coordination protocols in the research scenario, and cognitive planning improves milestone achievement rates by 3%. Code and datasets are public available at this https URL.
Comments: this https URL
Subjects: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY)
Cite as: arXiv:2503.01935 [cs.MA]
  (or arXiv:2503.01935v2 [cs.MA] for this version)
  https://doi.org/10.48550/arXiv.2503.01935

arXiv-issued DOI via DataCite

Submission history

From: Kunlun Zhu [view email]
[v1] Mon, 3 Mar 2025 05:18:50 UTC (5,997 KB)
[v2] Wed, 18 Nov 2026 17:51:17 UTC (2,005 KB)

来源:arXiv cs.CY · arxiv.org