arXiv cs.CL· Arush Tagade, Shaoheng Zhou, Jiaxin Wen, Shi Feng·· 10 小时前AI 评分45
自我建模干预可调节大语言模型的涌现性失准
Self-Modeling Interventions can Modulate Emergent Misalignment
AI 导读
研究提出自我建模干预（自我识别与自我报告），在 GPT-4.1、Qwen2.5-32B-Instruct、Seed-OSS-36B-Instruct 三个模型上验证其可逆转并预防涌现性失准（EM）。预防场景下，仅自我识别微调能持续降低失准且不恶化任何单项指标。实验还显示 EM 微调会使模型身份自我报告碎片化、自我识别性能降至接近随机水平，表明 EM 是对齐性格的失稳而非采纳连贯的失准人格。
正文 · 原文
Abstract:Emergent misalignment (EM) has been linked to the activation of misaligned persona vectors and evil character traits, suggesting that EM operates through disruption of the model's aligned character rather than direct learning of harmful content. Motivated by this connection, we study self-modeling interventions: specifically self-recognition and self-reports as character-targeted interventions that are distinct from existing in-training defenses. We conduct two-stage finetuning experiments across three models (GPT-4.1, Qwen2.5-32B-Instruct, Seed-OSS-36B-Instruct) and multiple EM datasets to compare self-recognition finetuning against benign finetuning baselines (correct domain-specific data, general knowledge, and word counting), finding it an effective defense in both reversal and prevention settings. For prevention, only self-recognition finetuning consistently reduces misalignment without exacerbating any individual metric, suggesting that character fortification specifically drives prevention. We also conduct in-training experiments across these models to compare the effectiveness of self-reports with benign interleaving and inoculation prompting. We provide further evidence for EM's relation to the LLM's default character by showing that EM finetuning leads to fragmentation of LLM's identity self-reports, reduces self-recognition performance to about chance, artificially corrupting self-recognition exacerbates misalignment caused by EM finetuning, and finetuning on self-reports from EM finetuned models can transfer misalignment. Together, these findings reframe EM not as the adoption of a coherent misaligned persona but as the destabilization of aligned character.
| Comments: | 18 pages, 11 figures. Accepted to NeurIPS 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.23700 [cs.CL] |
| (or arXiv:2606.23700v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.23700 arXiv-issued DOI via DataCite |
Submission history
From: Arush Tagade [view email]
[v1]
Thu, 4 Jun 2026 00:04:58 UTC (2,132 KB)
[v2]
Mon, 9 Nov 2026 22:42:24 UTC (2,896 KB)
来源：arXiv cs.CL · arxiv.org