ContinualSkillBench：LLM 智能体真能持续进化技能吗？

View PDF HTML (experimental)

Abstract:Modern agent frameworks equip large language models with external skill libraries to solve complex tasks. However, it remains unclear whether these systems can effectively evolve their skills and whether the resulting skills improve task-solving capabilities. To bridge this gap, we introduce ContinualSkillBench, a dynamic evaluation framework for in-context continual skill learning. It covers five domains, each with 100 tasks ordered by difficulty and opportunities for skill transfer. Our experiments show that sequential execution generally improves performance, but the gains vary substantially across models and domains. Moreover, across two models and three domains, retaining context and feedback without skill maintenance yields nearly the same mean performance as maintaining an explicit skill library. Explicit skills nevertheless provide selective benefits for tasks requiring reusable procedures or precise outputs. We further find that the weaker model (GPT-4o) accumulates more skills but reuses them less often, suggesting difficulty consolidating task-specific experience into reusable procedures. These findings show that current in-context skill evolution mechanisms can support continual adaptation, but still struggle to consistently consolidate experience into robust and transferable skills.

Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as:	arXiv:2608.03874 [cs.AI]
	(or arXiv:2608.03874v2 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2608.03874 arXiv-issued DOI via DataCite

Submission history

From: Tianyi Guan [view email]
[v1] Tue, 4 Aug 2026 16:15:02 UTC (2,870 KB)
[v2] Tue, 13 Oct 2026 12:38:59 UTC (766 KB)