arXiv cs.AI· Yuxuan Liu, Zhaochen Su, Hangyu Guo, Shijue Huang, Zhixiong Zhang, Kawai Chung, Zhongwei Xie, Tianshi Zheng, Wei Fan, Qing Zong, Haochen Shi, Baixuan Xu, Haoran Li, Yangqiu Song·· 3 天前AI 评分39
SkillHorizon：通过技能组合构建长时程智能体任务
SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
AI 导读
研究者提出 SkillHorizon 基准，通过技能组合在七种商业环境中构建长时程任务，任务依赖输入输出、共享资源与状态更新，参考交互负载从 64K 到 1M tokens。在 1,050 个任务上评测五个语言模型，平均局部目标完成率从 90.4% 降至 46.8%，负载越大智能体越常在工作未完成时提交失败答案。代码已开源。
正文 · 原文
Authors:Yuxuan Liu, Zhaochen Su, Hangyu Guo, Shijue Huang, Zhixiong Zhang, Kawai Chung, Zhongwei Xie, Tianshi Zheng, Wei Fan, Qing Zong, Haochen Shi, Baixuan Xu, Haoran Li, Yangqiu Song
Abstract:Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluation is to expand task requirements and their dependencies while keeping tasks executable and outcomes verifiable. We introduce SkillHorizon, a benchmark that constructs long-horizon tasks through skill composition in seven business environments. Its generators use domain-specific rules to turn reusable skill procedures into concrete business activities. These activities are connected through input-output dependencies, shared resources, and state updates, allowing workload and coordination demands to grow together. Successful reference executions validate task feasibility and measure interaction workload, while programmatic evaluators assess local completion and whole-task success. We evaluate five language models on 1,050 tasks with reference interaction workloads ranging from 64K to 1M tokens. Across these scales, mean local-goal completion falls from 90.4% to 46.8%, as completed work fails to keep pace with growing requirements. At larger workloads, agents more often submit unsuccessful final answers with substantial work unfinished and tool-call allowance remaining. Trace analysis also identifies cases where scripts execute successfully but their outputs fail downstream business checks. SkillHorizon provides a configurable testbed for studying sustained progress, dependency coordination, and complete task delivery in growing workflows. Our code is available at this https URL.
| Comments: | 29 pages, 7 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.14966 [cs.AI] |
| (or arXiv:2610.14966v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14966 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuxuan Liu [view email]
[v1]
Sat, 10 Oct 2026 14:36:39 UTC (474 KB)
来源：arXiv cs.AI · arxiv.org