arXiv cs.CL· Idan Simai, Shauli Ravfogel, Uri Shaham·· 12 小时前AI 评分31
基于条件自编码器的原则性非线性概念擦除与表征引导
Principled Nonlinear Concept Erasure and Steering
AI 导读
研究者提出一种基于条件自编码器的原则性框架，用于非线性概念擦除，在完美重建与总体独立性假设下可证明实现完美的非线性防护。该方法在多个数据集、敏感属性和语言模型上取得更优的效用-擦除权衡，无需目标任务标签即超越竞品的监督变体，且优化更简单、收敛更快。该框架还附带概念转换（表征引导）能力，推理时仅改变目标概念即可生成不同概念值的表征，无需额外训练成本。
正文 · 原文
Abstract:Learned representations often encode sensitive attributes, such as gender or race, that may be undesirable or unsafe to retain. Concept erasure aims to remove this information while preserving task-relevant content. Existing methods remain incomplete: linear methods provide guarantees only against linear probes, while nonlinear methods achieve stronger empirical erasure but lack formal guarantees and often sacrifice utility. In this work, we introduce a principled conditional autoencoder framework for nonlinear concept erasure. We follow the principle that erasure should remove only the sensitive attribute and nothing else, and we show that, under the stated assumptions, perfect reconstruction together with population-level independence is sufficient for perfect nonlinear guardedness, with preservation guarantees in the imperfect-reconstruction regime. Empirically, we attain a superior utility-erasure trade-off across multiple datasets, sensitive attributes, and language models, surpassing even the supervised variants of competing methods despite using no target-task labels, while being simpler to optimize and converging substantially faster. Beyond erasure, our framework yields concept conversion (representation steering) as a by-product: by changing only the target concept at inference time, it produces representations corresponding to different concept values - at no additional training cost - a capability no prior erasure method provides, and on which we likewise demonstrate strong empirical performance.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.32516 [cs.LG] |
| (or arXiv:2610.32516v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32516 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Idan Simai [view email]
[v1]
Fri, 23 Oct 2026 11:44:02 UTC (137 KB)
来源：arXiv cs.CL · arxiv.org