arXiv cs.CL· Xunlei Chen, Qinghui Gong, Jingkun Xue, Qihe Liu, Shijie Zhou, Fei Ye·· 9 天前AI 评分22
并非所有变化都必要：大语言模型遗忘中的可恢复漂移
Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning
AI 导读
研究者提出 Propose-Then-Project Unlearning（PTP-U）框架，先通过局部解析编辑削弱目标知识关联，再将非目标输出分布与原模型对齐，在固定遗忘约束下恢复能力。在三个基准上，PTP-U 实现 81.22%-91.03% 的遗忘率，平均保留 94.20% 的非目标效用，在同等遗忘水平下持续保持更高的非目标效用。
正文 · 原文
Abstract:Machine unlearning in large language models aims to remove unwanted knowledge while preserving the model's remaining capabilities. Although existing methods use retention objectives or restrict where edits occur, achieving the desired forgetting level can still leave collateral changes that impair non-target behavior. Our recovery comparisons suggest that some of these changes can be reversed while preserving observed forgetting performance. In this work, we present Propose-Then-Project Unlearning (PTP-U), a framework that combines targeted forgetting with the recovery of non-target capabilities. PTP-U first applies local analytic edits to weaken target knowledge associations, then aligns non-target output distributions with those of the original model to recover capabilities while maintaining fixed forgetting constraints. Both stages serve a common goal: satisfying the forgetting requirements while preserving fluent generation and performance on non-target tasks. Across three benchmarks, PTP-U achieves the strongest forgetting-retention trade-off among evaluated methods, reaching 81.22%-91.03% forgetting while preserving 94.20% non-target utility on average. At matched forgetting, PTP-U consistently retains higher non-target utility.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.11915 [cs.CL] |
| (or arXiv:2610.11915v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11915 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xunlei Chen [view email]
[v1]
Thu, 8 Oct 2026 13:12:40 UTC (4,829 KB)
来源：arXiv cs.CL · arxiv.org