arXiv cs.AI· Yu Cheng, Dehai Zhao, Zhongxin Liu, Qing Huang, Zhenchang Xing, Xiaoxue Ren·· 9 天前AI 评分27
Agent Skills 下游效用实证研究：同一技能可助益也可拖累任务表现
An Empirical Study of Agent Skills' Downstream Utility
AI 导读
一项针对 87 个 SkillsBench 任务的实证研究显示，同一 Agent Skills 在 36.78% 的任务上对部分配置有帮助、对另一些配置反而有害，推荐流程可能变成执行负担。
正文 · 原文
Abstract:Agent Skills package procedural guidance and resources for reuse, but a relevant Skill does not necessarily improve task performance. Existing studies characterize Skill content and evaluate downstream performance, yet provide limited explanations of how utility depends on content, execution configuration, and multi-Skill organization. We conduct an empirical study on 87 SkillsBench tasks, defining downstream utility as the pass-rate difference from No-Skill on the same tasks under the same model--harness configuration. We compare the same Skills across nine configurations, then examine alternative published Skills and organizations of fixed Skill sets under three selected configurations. We retrieve marketplace candidates from a curated corpus of 37,596 Skills. LLM-assisted analysis of content, execution traces, and final artifacts, followed by author review, relates provided support to actual use and task outcomes. The same Skills help some configurations and hurt others on 36.78\% of tasks, with trajectories showing that recommended procedures can become an execution burden. Relevance rankings overlook more useful candidates. Within the evaluated candidate sets, reranking by support for required operations raises first-choice pass rates by 4.35--5.80 percentage points across the three configurations. We derive 17 authoring practices linking executable procedures to recovery, preservation of task requirements, and checks on final artifacts. Stage Plan and Dependency DAG outperform use order alone, with DAG's additional benefits concentrated in tasks supplied with five or six Skills. These findings guide developers to assess usable operation support, allow procedure adaptation while preserving task requirements, and make artifact dependencies explicit when organizing Skills.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.08875 [cs.AI] |
| (or arXiv:2610.08875v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08875 arXiv-issued DOI via DataCite |
Submission history
From: Yu Cheng [view email]
[v1]
Tue, 6 Oct 2026 06:05:00 UTC (16,921 KB)
[v2]
Thu, 8 Oct 2026 04:06:52 UTC (16,921 KB)
来源：arXiv cs.AI · arxiv.org