arXiv cs.CL· Jaewon Cho, Minseo Yoon, Jinyoung Park, Jinwoo Seo, Jaewon Chu, Alice Oh, Hyunwoo J. Kim·· 11 小时前AI 评分26
TOPSD：用可教学前缀解决多语言推理蒸馏中的 Teacher Drift
Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation
AI 导读
针对 On-Policy Self-Distillation 在多语言推理中出现的 Teacher Drift 现象——教师模型会跟随学生的错误中间步骤而非纠正，且低资源语言中漂移出现更早，研究者提出 TOPSD，用师生似然估计漂移起点并选取前缀，再验证教师能否从该前缀恢复正确答案，仅对通过验证的前缀做蒸馏。
正文 · 原文
Abstract:On-Policy Self-Distillation (OPSD) improves reasoning by training a student on its own reasoning, using the same model conditioned on reference solutions as a teacher. Applying OPSD to multilingual reasoning requires the teacher to provide useful feedback across languages, even when the student makes mistakes in intermediate steps. We observe a phenomenon we call Teacher Drift: as these mistakes accumulate, the teacher may follow the student's incorrect steps instead of correcting them. Our analysis shows that this shift tends to occur earlier in lower-resource languages. To account for these differences across languages, we propose Teachable On-Policy Self-Distillation (TOPSD), which determines where and whether teacher supervision should be applied to each reasoning trajectory. TOPSD uses teacher-student likelihood to estimate where drift begins and select a prefix for supervision. It then verifies whether the teacher can recover the correct answer from that prefix, applying distillation only to prefixes that pass verification. Experiments on multilingual mathematical reasoning across ten languages and three model scales show that TOPSD consistently outperforms conventional OPSD, particularly in low- and medium-resource languages.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.08179 [cs.CL] |
| (or arXiv:2611.08179v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08179 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jaewon Cho [view email]
[v1]
Thu, 5 Nov 2026 15:34:23 UTC (1,658 KB)
来源：arXiv cs.CL · arxiv.org