arXiv cs.CL· Leon Sick, Lukas Hoyer, Yannick Str\"umpler, Christian Reisswig, Timo Ropinski, Federico Tombari·· 12 小时前AI 评分40
无需外部教师：LLM 通过在线策略自训练实现无数据自我提升
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
AI 导读
研究提出将在线策略自蒸馏重构为自训练框架，模型仅凭粗粒度的高层主题描述即可自行合成问题与特权推理演示，无需人工标注的问答对或提示词。该方法在文本与多模态领域均实现持续自我提升，在标准语言基准和视觉问答（VQA）上表现一致，且在未自训练领域保留原有能力。
正文 · 原文
Abstract:On-policy self-distillation enables post-training foundation models without relying on stronger external teachers. However, existing approaches rely on ground-truth question-answer pairs, human-curated problem sets, or other supervised signals. In this work, we reformulate on-policy self-distillation as a self-training framework, driven by self-generated question-answer pairs and privileged reasoning demonstrations. We push supervision to the limit: guided solely by coarse, high-level topic descriptions, the model synthesizes its own curriculum of queries and privileged reasoning traces without access to any human-curated prompts or answers. We find that self-distillation on fully synthetic privileged information enables a model to improve across multiple domains while retaining its capabilities on domains it is not self-trained on. We demonstrate the generality of this framework across text and multimodal domains, achieving consistent self-improvement on standard language benchmarks and Visual Question Answering. Our findings demonstrate that foundation models possess rich latent capabilities that can be amplified without external task supervision or curated problem distributions.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.32770 [cs.CL] |
| (or arXiv:2610.32770v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32770 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Leon Sick [view email]
[v1]
Fri, 23 Oct 2026 15:12:03 UTC (152 KB)
来源：arXiv cs.CL · arxiv.org