arXiv cs.AI· Xuanfeng Yang, Le Gui, Jiaming Wang, Xuan Zhang·· 12 小时前AI 评分22
CA-MOPD：能力感知的多教师在线策略蒸馏方法
Beyond Policy Discrepancy: Capability-Aware Multi-Teacher On-Policy Distillation
AI 导读
研究者提出能力感知多教师在线策略蒸馏方法 CA-MOPD，先合并强化学习专家得到强学生初始化，再用验证器估计的教师—学生能力差距调整各领域蒸馏强度。在数学、代码生成和指令遵循的六个基准上，CA-MOPD 平均得分 32.22，分别超过朴素 MOPD、Open-MOPD 和合并初始化 0.86、0.83 和 0.74 分。
正文 · 原文
Abstract:Multi-teacher on-policy distillation (MOPD) integrates domain specialists into a shared student, but determining how strongly each teacher should supervise remains challenging. Existing discrepancy-based allocation can assign strong supervision even when the student already matches or exceeds the teacher's task performance, since behavioral mismatch does not necessarily imply a remaining capability deficit. We propose Capability-Aware Multi-Teacher On-Policy Distillation (CA-MOPD), which combines specialist merging with capability-guided allocation to account for what the student has already learned. CA-MOPD first merges reinforcement-learning specialists into a strong student initialization, then uses verifier-estimated teacher--student capability gaps to adjust domain-level distillation strength. Supervision decreases as capability gaps close and increases when estimated student capability regresses, while dense token-level guidance remains provided by the teachers. Across six benchmarks spanning mathematics, code generation, and instruction following, CA-MOPD achieves an average score of 32.22, outperforming naive MOPD, Open-MOPD, and the merged initialization by 0.86, 0.83, and 0.74 points, respectively. Controlled comparisons further support the benefits of merged initialization and capability-gap guidance under matched training budgets.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.26950 [cs.LG] |
| (or arXiv:2610.26950v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.26950 arXiv-issued DOI via DataCite |
Submission history
From: Xuanfeng Yang [view email]
[v1]
Sun, 18 Oct 2026 09:09:06 UTC (166 KB)
来源：arXiv cs.AI · arxiv.org