arXiv cs.CL· Florian Mai, David Kacz\'er·· 2 天前AI 评分34
ContextLift：打破条件性涌现错位背后的伪相关
ContextLift: Breaking the Spurious Correlation Behind Conditional Emergent Misalignment
AI 导读
针对语言模型在窄域错位数据上微调后出现的条件性涌现错位，研究者提出反事实增强方法 ContextLift：将一半安全数据行复制并逐字插入随机训练提示中的上下文句，对齐答案不变，从而打破上下文与错位答案的伪相关。
正文 · 原文
Abstract:Finetuning a language model on narrowly misaligned data can make it broadly misaligned. Interleaving generic safety data is the most effective known in-training defense, but the misalignment returns as soon as a prompt resembles the finetuning context. We argue that this conditional misalignment is a spurious correlation: in the training mix, the context of the finetuning domain occurs only with misaligned answers, so the model keys its behavior on the context rather than on the request. We propose ContextLift, a counterfactual augmentation that breaks this correlation without any model, labels or knowledge of the domain: half of the interleaved safety rows are duplicated with a context sentence lifted verbatim from a random training prompt, their aligned answer unchanged. On Qwen2.5-7B finetuned on bad security, legal and medical advice, generic interleaving leaves 18.9%, 41.9% and 58.2% misaligned answers under a domain cue; ContextLift reduces this to 2.6%, 6.1% and 12.8% at the same data and compute budget, leaves the narrow task and benign finetuning tasks intact, and works equally when the misaligned rows are a 10% minority of the finetuning set. Code: this https URL
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.21633 [cs.CL] |
| (or arXiv:2610.21633v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21633 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: David Kaczér [view email]
[v1]
Thu, 15 Oct 2026 10:18:41 UTC (29 KB)
来源：arXiv cs.CL · arxiv.org