arXiv cs.AI· Haichen Hu, Yuheng Zhang, David Simchi-Levi·· 14 小时前AI 评分29
IterCaL：在无目标域奖励反馈下缓解 LLM 蒸馏中的教师偏差
Iterative Calibration and Learning: Mitigating Teacher Bias in LLM Distillation without Target-Domain Reward Feedback
AI 导读
研究者提出 IterCaL（Iterative Calibration and Learning），通过 token 级分支交替进行源奖励教师校准与学生在目标提示词上的学习，在缺少目标域奖励反馈时缓解 LLM 蒸馏中的教师偏差传播。
正文 · 原文
Abstract:Large language model (LLM) distillation can propagate teacher bias under covariate shift when target reward feedback is unavailable. We propose Iterative Calibration and Learning (IterCaL), which alternates source-reward teacher calibration through token-level branching with student learning on target prompts. We prove polynomial convergence in expected average KL to the oracle student, the reference-regularized target-reward maximizer within a potentially misspecified student class; a quantified source-to-target calibration discrepancy gives an explicit additive true-return term. Our separation shows that regularized direct teacher matching retains positive oracle-KL error at its population optimum for every shared reference penalty, even when the teacher is stronger than every student. On the same instance, IterCaL converges to the oracle using source rewards alone.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Statistics Theory (math.ST); Machine Learning (stat.ML) |
| Cite as: | arXiv:2609.17474 [cs.LG] |
| (or arXiv:2609.17474v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.17474 arXiv-issued DOI via DataCite |
Submission history
From: Haichen Hu [view email]
[v1]
Tue, 15 Sep 2026 17:15:40 UTC (102 KB)
[v2]
Thu, 29 Oct 2026 03:09:44 UTC (122 KB)
来源：arXiv cs.AI · arxiv.org