arXiv cs.CL· Haoyu Sun, Wenxuan Wang, Mingyang Song, Jujie He, Weinan Zhang, Yang Liu, Yang Yang, Yu Cheng·· 13 小时前AI 评分36
Agent Planning Benchmark：面向 LLM 智能体规划能力的诊断框架
Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agents
AI 导读
研究者推出 Agent Planning Benchmark（APB），一个专门诊断 LLM 智能体规划能力的基准，含 22 个领域、5 种设置下的 4,209 个多模态案例，覆盖整体规划、反馈条件下的分步规划，以及多余工具、损坏工具和不可解任务下的鲁棒性。
正文 · 原文
Abstract:Planning is central to LLM agents: before acting, an agent must decompose goals, select tools, reason over constraints, and decide when a task is infeasible. Yet existing agent evaluations often report only end-to-end success, making it difficult to determine whether failures stem from planning or execution. We introduce Agent Planning Benchmark (APB), a planning-specific diagnostic benchmark with 4,209 multimodal cases across 22 domains and five settings, covering holistic planning, feedback-conditioned step-wise planning, and robustness under extraneous tools, broken tools, and unsolvable tasks. Across 12 MLLMs, APB reveals systematic weaknesses in long-horizon planning, tool-noise robustness, calibrated refusal, and inference-time refinement. We further validate APB on 200 ToolSandbox tasks and 200 $\tau^2$-bench tasks, where APB-guided refinement consistently improves plan correctness, plan grade, and downstream execution metrics across three representative models. APB thus serves as an upstream diagnostic complement to execution benchmarks. The APB benchmark and code are available in \href{this https URL}{this URL}.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.04874 [cs.CL] |
| (or arXiv:2606.04874v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.04874 arXiv-issued DOI via DataCite |
Submission history
From: Haoyu Sun [view email]
[v1]
Wed, 3 Jun 2026 13:37:47 UTC (4,524 KB)
[v2]
Fri, 5 Jun 2026 05:23:52 UTC (4,524 KB)
[v3]
Mon, 5 Oct 2026 09:18:53 UTC (4,541 KB)
[v4]
Tue, 27 Oct 2026 07:02:03 UTC (4,539 KB)
来源：arXiv cs.CL · arxiv.org