首个智能体蒸馏统一综述：跨模型、制品与执行框架的知识迁移

Authors:Ziluowen Luo, Senzhang Wang, Chaozhuo Li, Jun Yin, Hao Yan, Ming Cheng, Chenxu Wang, Songyang Liu, Litian Zhang, Qiwei Ye, Zheng Liu, Chengqi Zhang, Philip S. Yu

View PDF HTML (experimental)

Abstract:Knowledge distillation provides a teacher-student framework for transferring capabilities at reduced cost and has been widely studied for large language models (LLMs). As LLM-based agents move from response generation to sustained, closed-loop task solving, distillation expands beyond model parameters to knowledge retained in reusable resources and execution workflows. This shift from model-level to agent-level knowledge transfer supports efficient deployment, broader capability access, and continual self-evolution. However, existing surveys largely examine model distillation and agent system design separately, limiting comparison of transfer strategies across agent components. To bridge this gap, we formalize Agent Distillation as persistent knowledge transfer between teacher and student agents and present its first unified survey. Distinguishing experience used for transfer from knowledge retained for reuse, we develop a knowledge-substrate taxonomy based on where knowledge persists: models, artifacts, or harnesses. Building on this taxonomy, we analyze cross-substrate transfer paths and hybrid strategies. To assess the resulting capabilities, we review benchmarks and evaluation protocols for retention, causal attribution, and deployed utility without further teacher support. This survey provides a unified basis for studying agent knowledge transfer and developing efficient, adaptable, and self-evolving agents.

Submission history

From: Ziluowen Luo [view email]
[v1] Tue, 29 Sep 2026 03:38:17 UTC (5,233 KB)
[v2] Tue, 10 Nov 2026 08:47:08 UTC (3,813 KB)