arXiv cs.AI· Heng Li, Yong Zhang, Ning Cheng, Zhigen Li, Yun Zhu, Yanmeng Wang, Shaojun Wang, Jing Xiao·· 9 天前AI 评分21
PIVOT：基于困惑度的 KD-to-RL 动态切换调度，用于垂直领域小样本蒸馏
PIVOT: Perplexity-Informed KD-to-RL Transition Scheduling for Vertical-Domain Few-Shot Distillation
AI 导读
PIVOT 是一种动态切换框架，依据教师模型评估的序列困惑度在 OPD 与 GRPO 之间路由样本：低困惑度样本转入 GRPO 做奖励驱动精炼，高困惑度样本留在 OPD 继续获取领域知识。
正文 · 原文
Abstract:Vertical-domain few-shot classification remains challenging for small language models, as limited supervision makes it difficult to acquire domain-specific decision knowledge. On-Policy Distillation (OPD) can improve teacher-guided adaptation by supervising student-generated rollouts, while GRPO-based reinforcement learning can further refine downstream predictions. However, existing KD-to-RL pipelines typically rely on globally fixed transition schedules, ignoring that different samples may require different amounts of teacher-guided acquisition before reward-driven refinement. We propose PIVOT (Perplexity-Informed Transition Optimization), a dynamic transition framework that routes samples between OPD and GRPO according to teacher-evaluated sequence perplexity. PIVOT moves low-perplexity samples to GRPO for reward-driven refinement while keeping high-perplexity samples under OPD for continued domain knowledge acquisition. Experiments on Banking77 and HWU64 show that PIVOT consistently outperforms continued OPD and globally synchronized OPD$\rightarrow$GRPO baselines under the same number of post-warm-up student optimization steps, achieving stronger downstream performance and more stable training dynamics.
| Comments: | Accepted to Findings of EMNLP 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.11167 [cs.LG] |
| (or arXiv:2610.11167v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11167 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yong Zhang [view email]
[v1]
Thu, 8 Oct 2026 03:24:55 UTC (2,250 KB)
来源：arXiv cs.AI · arxiv.org