arXiv cs.CL· Bingqi Shang, Yang Zhang, Yiwei Chen, Changsheng Wang, Shiyu Chang, Bingquan Shen, Sijia Liu·· 12 小时前AI 评分34
原生安全能否迁移？通过 on-policy 蒸馏跨模型复用安全预训练
Can Native Safety Travel? Reusing Safety Pretraining Across Models via On-Policy Distillation
AI 导读
研究提出“原生安全复用”问题：能否将小型安全预训练“教师”模型的原生安全能力迁移到标准预训练的“学生”模型。方法采用 on-policy 蒸馏，学生生成安全提示词回复，教师对其 token 级预测进行监督，并通过跨 tokenizer 对齐实现跨模型族迁移。
正文 · 原文
Abstract:Safety pretraining has emerged as a promising paradigm for embedding safety deeply into a model during pretraining, yielding robustness to subsequent fine-tuning that conventional safety post-training often lacks. We refer to this safety acquired during model pretraining as native safety. Yet native safety is expensive to acquire, confined to the model in which it was pretrained, and often accompanied by lower general utility. In this work, we rethink the practical value of safety-pretrained models and introduce, for the first time, the native safety reuse problem: Can native safety be transferred from a specialized, potentially smaller, safety-pretrained "teacher" to standard-pretrained "students," enabling them to inherit its robustness while preserving their general utility? We show that native safety can "travel" through on-policy distillation. A standard-pretrained student generates responses to safety-pretraining prompts, while a small safety-pretrained teacher supervises the student's token-level predictions along its own generated trajectories; cross-tokenizer alignment further enables transfer across model families. The resulting students exhibit substantially greater safety robustness than their safety-post-trained counterparts while retaining higher general utility than the teacher: subsequent fine-tuning raises the average attack success rate (ASR) of the 1.7B student by only 5.00%, compared with 11.58% for safety post-training, and its average utility of 48.20% far exceeds the aligned teacher's 27.42%. Unlike conventional large-to-small knowledge distillation, our framework enables selective, small-to-large safety transfer: it reuses the robust safety expertise of a smaller pretrained teacher without inheriting its utility limitations. These findings establish native safety as a reusable model property rather than one permanently tied to the originally pretrained model.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.27555 [cs.LG] |
| (or arXiv:2610.27555v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27555 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bingqi Shang [view email]
[v1]
Tue, 20 Oct 2026 03:13:04 UTC (295 KB)
来源：arXiv cs.CL · arxiv.org