arXiv cs.AI· Zijie Liu, Qi Yang Huang, Jinhao Duan, Yueh Che Lee, Thomas A. Joseph, Zayne Turner, Adam Seligman, Tianlong Chen, Zhaozhuo Xu·· 12 小时前AI 评分38
AgentOrch 基准：全局编排与逐步交互的准确率差距有多大？
AgentOrch: How Close Can Global Orchestration Come to Step-by-Step Interaction?
AI 导读
AgentOrch 基准在 650 个企业 SOP 和 3,220 个运行时变体上对比全局编排与逐步交互，结果显示逐步交互在所有评测模型上仍更准确，但差距从早期模型到最强近期模型已缩小约 79%。增量式工作流构建平均提升全局编排准确率 9.98 个百分点，用执行反馈修订完整工作流在留出运行时变体上提升 6.13 个百分点，针对完整工作流的目标训练在五个模型上平均提升 12.18 个百分点。
正文
Abstract:Enterprise workflow agents typically rely on step-by-step interaction (Step-by-Step): the agent executes an action, observes its result, and invokes a large language model (LLM) again to decide the next step. Global orchestration (Global) instead generates a complete executable workflow in advance and executes it without further LLM calls, substantially reducing inference overhead, but it remains less accurate than Step-by-Step. We study how close Global can come to Step-by-Step and what narrows the remaining gap. We introduce AgentOrch, a benchmark that directly compares global orchestration with step-by-step interaction on 650 enterprise SOPs and 3,220 runtime variants under matched tools, environments, and evaluation. Step-by-Step remains more accurate for every evaluated model, but the gap has shrunk by about 79% from early models to the strongest recent model. We identify three routes to further narrowing this gap. Incremental workflow construction improves Global accuracy by an average of 9.98 percentage points (pp) without runtime execution feedback. Revising complete workflows with execution feedback improves all evaluated models, including 6.13 pp on held-out runtime variants. Targeted training on complete workflows improves accuracy by an average of 12.18 pp across all five trained models. Although a gap remains, global orchestration offers a promising path toward enterprise agents that deliver higher accuracy with fewer LLM calls.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33375 [cs.AI] |
| (or arXiv:2610.33375v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33375 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zijie Liu [view email]
[v1]
Sat, 24 Oct 2026 00:08:11 UTC (1,532 KB)
来源：arXiv cs.AI · arxiv.org