跳到正文
arXiv cs.AI· Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Gu Simiu, Sulong Xu, Cong Fang·· 10 小时前AI 评分42

COPD:通过自然语言沟通从黑盒教师模型蒸馏长程任务能力

Communicative On-Policy Distillation: Distilling Black-Box Teacher Capabilities through Communication for Long-Horizon Tasks

AI 导读

研究者提出 Communicative On-Policy Distillation(COPD),以自然语言沟通为媒介,将更强黑盒教师模型的能力蒸馏到更小的学生模型,学生可自主决定何时向教师提问且无需预设模板。

正文 · 原文

Authors:Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Gu Simiu, Sulong Xu, Cong Fang

View PDF

Abstract:On-policy distillation (OPD) is a powerful paradigm for improving language-model agents on long-horizon tasks, yet its conventional formulation relies on teacher logits that text-only black- box models do not expose. To address this limitation, we introduce Communicative On-Policy Distillation (COPD), which uses natural-language communication as a medium for distilling a stronger model's capabilities into a smaller student. During task execution, the student au- tonomously decides when to consult a black-box teacher and formulates open-ended natural- language requests without predefined question templates or intervention schedules. Successful student-teacher interactions are retained as privileged references to supervise fresh adviser-free student rollouts through on-policy self-distillation, jointly with task-reward reinforcement learning. On CAMG-II, COPD achieves 2.46 times the gain of the strongest black-box OPD baseline. The capabilities it learns through communication generalize to Terminal-Bench 2.1 and SWE-Bench Pro V2, improving task success without adviser access.
Comments: 13 pages, 8 figures, 2 tables. Project page: this https URL
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2611.23673 [cs.LG]
  (or arXiv:2611.23673v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.23673

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Lirui Luo [view email]
[v1] Wed, 18 Nov 2026 00:56:10 UTC (10,755 KB)

来源:arXiv cs.AI · arxiv.org