arXiv cs.AI· Woosung Koh, Hyunsoo Lee, Kyungjae Lee, Haeju Park, Dahyun Lee, Dasol Hwang, Moontae Lee·· 6 天前AI 评分24
SynthSelf：用合成自蒸馏实现小模型持续后训练
Continually Post-training Small Models via Synthetic Self-distillation
AI 导读
针对 SDFT 依赖强上下文学习能力、在小模型上失效的问题，研究者提出合成自蒸馏（SynthSelf），将分布内改写任务外包给同族、略大的对齐模型，并结合离策略监督与拒绝采样。在 4 个能力获取数据集和 14 个保持性基准上，SynthSelf 在 3 种小模型规模下均优于包括离策略与在策略蒸馏在内的 6 种方法。
正文 · 原文
Abstract:Continual post-training faces a dual challenge of acquisition and retention: models must learn new capabilities while preserving prior knowledge. While Self-Distillation Fine-Tuning (SDFT) mitigates this using self-supervision with in-context demonstrations, it relies on strong in-context learning and instruction-following abilities. Consequently, SDFT fails on smaller models critical for low-latency, on-device use. To enable continual learning at smaller scales, we first identify the root cause of failure. We show that poor acquisition and forgetting correlate linearly with the post-trained model's distribution shift, revealing the necessity of in-distribution learning. We thus introduce Synthetic Self-Distillation (SynthSelf), imitating SDFT's in-distribution rewrite by outsourcing the task to a minimally larger, same-family aligner. Coupled with off-policy supervision and rejection sampling, this provides the high-quality supervision small models require without breaking their original distribution. Evaluations spanning 4 acquisition datasets (including a mixture) and 14 retention benchmarks show SynthSelf consistently outperforms 6 methods (including off- and on-policy distillation) across 3 small model scales. Extensive further analyses confirm SynthSelf paves the way toward effective small-model continual learning.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.13113 [cs.LG] |
| (or arXiv:2610.13113v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13113 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Woosung Koh [view email]
[v1]
Fri, 9 Oct 2026 07:06:34 UTC (500 KB)
来源：arXiv cs.AI · arxiv.org