arXiv cs.AI· Wei Zhai, Xiang Liu, Qiang Huang, Rui Qian, Lemao Liu, Ziwei Li, Ziqi Wang, Zhitao Huang, Dejing Dou·· 9 天前AI 评分27
Nullify：面向 LLM 免训练遗忘的零空间激活引导方法
Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning
AI 导读
研究者提出 Nullify，一种免训练、非破坏性的激活引导方法，用于 LLM 遗忘：在推理时用引导向量把隐私相关激活重定向，远离模型记忆的答案，同时满足零空间约束，使保留查询的激活基本不受影响。在 TOFU 和 MUSE 上，Nullify 的遗忘质量达到或超过现有基线，并近乎无损地保持模型效用。由于完全无需更新权重，它可作为即插即用的推理时干预框架。
正文 · 原文
Abstract:Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to prevent privacy leakage with minimal loss of model utility. However, existing methods struggle to balance forget quality with utility, and typically incur substantial computational costs due to parameter fine-tuning. To address this, we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning. Nullify employs steering vectors during inference to redirect privacy-related activations away from their memorized answers, while satisfying a null-space constraint that leaves retained-query activations essentially unaffected to maintain utility. Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility. By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2610.10655 [cs.LG] |
| (or arXiv:2610.10655v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10655 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wei Zhai [view email]
[v1]
Wed, 7 Oct 2026 16:24:07 UTC (80 KB)
来源：arXiv cs.AI · arxiv.org