arXiv cs.AI· Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Gu Simiu, Sulong Xu, Cong Fang·· 10 小时前AI 评分42
COPD:通过自然语言沟通从黑盒教师模型蒸馏长程任务能力
Communicative On-Policy Distillation: Distilling Black-Box Teacher Capabilities through Communication for Long-Horizon Tasks
AI 导读
研究者提出 Communicative On-Policy Distillation(COPD),以自然语言沟通为媒介,将更强黑盒教师模型的能力蒸馏到更小的学生模型,学生可自主决定何时向教师提问且无需预设模板。
正文 · 原文
Authors:Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Gu Simiu, Sulong Xu, Cong Fang
Abstract:On-policy distillation (OPD) is a powerful paradigm for improving language-model agents on long-horizon tasks, yet its conventional formulation relies on teacher logits that text-only black- box models do not expose. To address this limitation, we introduce Communicative On-Policy Distillation (COPD), which uses natural-language communication as a medium for distilling a stronger model's capabilities into a smaller student. During task execution, the student au- tonomously decides when to consult a black-box teacher and formulates open-ended natural- language requests without predefined question templates or intervention schedules. Successful student-teacher interactions are retained as privileged references to supervise fresh adviser-free student rollouts through on-policy self-distillation, jointly with task-reward reinforcement learning. On CAMG-II, COPD achieves 2.46 times the gain of the strongest black-box OPD baseline. The capabilities it learns through communication generalize to Terminal-Bench 2.1 and SWE-Bench Pro V2, improving task success without adviser access.
| Comments: | 13 pages, 8 figures, 2 tables. Project page: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.23673 [cs.LG] |
| (or arXiv:2611.23673v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23673 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lirui Luo [view email]
[v1]
Wed, 18 Nov 2026 00:56:10 UTC (10,755 KB)
来源:arXiv cs.AI · arxiv.org