arXiv cs.AI· Jiaqi Tian, Ting Long·· 6 天前AI 评分22
通过表征几何正则化缓解 LLM 自我改进中的能力退化
Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization
AI 导读
针对 On-Policy Self-distillation（OPSD）自我改进方法会逐步退化模型已有能力的问题，研究者提出表征几何正则化（RGR），通过增强簇内紧凑性与簇间分离度来保持隐藏表征几何。RGR 可无缝集成进现有 OPSD 方法而不改变其核心优化流程，在多种 OPSD 方法与基准上普遍缓解了能力退化，同时保留能力提升。代码已开源。
正文 · 原文
Abstract:On-Policy Self-distillation is a self-improvement paradigm that enables large language models (LLMs) to improve their capabilities by leveraging the same model as both teacher and student under different conditioning contexts. However, we find that OPSD methods can progressively degrade previously acquired capabilities: queries that are correctly responded at earlier training stages may become incorrectly responded as optimization proceeds, even the overall performance continues to improve. Further analysis reveals that this degradation coincides with a deterioration of the geometry of hidden representations, characterized by reduced intra-cluster compactness and diminished inter-cluster separation. Motivated by this observation, we propose Representation Geometry Regularization (RGR), which preserves representation geometry during post-training by encouraging compactness within cluster and separation across cluster. RGR can be seamlessly integrated into existing OPSD methods without altering their core optimization procedures. Extensive experiments on multiple OPSD methods and benchmarks show that RGR generally alleviates capability degradation while retaining capability improvements. Our code is available in this https URL.
| Comments: | 15 pages, 5 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.12855 [cs.AI] |
| (or arXiv:2610.12855v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12855 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiaqi Tian [view email]
[v1]
Fri, 9 Oct 2026 01:28:50 UTC (1,390 KB)
来源：arXiv cs.AI · arxiv.org