arXiv cs.AI· Shi-Yu Tian, Yong Liu, Ximan Liu, Tianrui Zhu, Sule Bai, Xinwei Yang, Ruiqi Liu, Kun-Yang Yu, Ming Yang, Zhi Zhou, Yu-Feng Li, Yipeng Sun, Gu Simiu·· 6 天前AI 评分22
多教师在线策略蒸馏的跨教师守护方法 Road MOPD
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
AI 导读
针对多教师在线策略蒸馏（MOPD）中域路由无法避免共享参数上的梯度冲突问题，研究者提出 Road MOPD，引入非路由教师作为跨域评论者，通过 token 级信用分配与多教师梯度投影协调监督。实验显示，该方法在高冲突场景下提升超过 5 个百分点，并改善低冲突与三域设置下的整体表现。
正文 · 原文
Authors:Shi-Yu Tian, Yong Liu, Ximan Liu, Tianrui Zhu, Sule Bai, Xinwei Yang, Ruiqi Liu, Kun-Yang Yu, Ming Yang, Zhi Zhou, Yu-Feng Li, Yipeng Sun, Gu Simiu
Abstract:Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) by routing domain-specific data to specialized teachers, integrating their capabilities through dense token-level supervision on student-generated trajectories. However, domain routing only distinguishes the supervision sources for different domains; updates from all domains still act on shared parameters. When teachers differ substantially in response style and domain expertise, supervision that is effective in isolation may still lead to persistent gradient conflicts and capability degradation during joint optimization. To address this challenge, we propose Cross-Teacher Guarding for MOPD (Road MOPD), which introduces non-routed teachers as auxiliary cross-domain critics on the same student trajectories and coordinates multi-teacher supervision through token-level credit assignment and gradient updates. Specifically, cross-teacher-guided credit assignment uses teacher agreement to strengthen token-level corrections jointly supported by the teachers and attenuate conflicting signals; preventive multi-teacher gradient projection uses auxiliary gradients from the same trajectories to constrain potentially conflicting updates and adaptively controls projection strength to retain target-domain learning signals. In experiments, Road MOPD mitigates degradation in the disadvantaged domain, gains over 5 percentage points under high conflict, and improves overall performance in low-conflict and three-domain settings.
| Comments: | 30 pages, 9 figures, 10 tables, including references and appendices |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.13521 [cs.AI] |
| (or arXiv:2610.13521v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13521 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shi-Yu Tian [view email]
[v1]
Fri, 9 Oct 2026 12:41:51 UTC (1,102 KB)
来源：arXiv cs.AI · arxiv.org