arXiv cs.CL· Baohang Li, Xiaocheng Feng, Bing Qin·· 11 小时前AI 评分34
TandemBench：多智能体协作决策模型基准测试
TandemBench: Benchmarking Decision Models for Multi-Agent Collaboration
AI 导读
研究者推出 TandemBench，包含 1200 道多智能体烹饪场景问题，用于评测决策模型在团队分工与并行行动上的选择能力。评测覆盖 40 个原生决策模型与 10 个通用大语言模型，最佳原生决策模型决策准确率为 62.50%，但在最优性验证上接近随机水平。配对测试显示部分模型无论是否已给出最优方案都会声称存在更优替代，说明单看决策准确率会掩盖系统性判断错误。
正文 · 原文
Abstract:Decision models choose among predefined options in tasks such as routing and action selection. In multi-agent collaboration, a decision determines how members divide work, exchange resources, and act in parallel. We introduce TandemBench, a benchmark of 1,200 questions in multi-agent cooking scenarios where coordination choices affect goal completion. Models receive complete global information and candidate team plans, with reference answers established through execution and exact search. Models select an optimal team plan and, in paired variants that retain or remove it, judge whether better arrangements exist beyond the supplied options. We evaluate 40 native decision models and compare them with 10 general-purpose language models. The best native decision model achieves 62.50% decision accuracy. Native decision models remain close to chance in optimality verification. Paired tests reveal that some report better alternatives regardless of whether an optimal plan is already present. These findings show that team-plan decision accuracy alone can conceal systematic errors in judging whether the supplied options contain an optimal arrangement. The dataset and evaluation code are publicly available at this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.05827 [cs.CL] |
| (or arXiv:2611.05827v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05827 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Baohang Li [view email]
[v1]
Wed, 4 Nov 2026 02:08:57 UTC (1,622 KB)
来源：arXiv cs.CL · arxiv.org