arXiv cs.AI· Rose N. Kwon, Samy Mammeri, Sabyasachi Sahoo, Christian Gagn\'e, Thomas Jiralerspong·· 23 小时前AI 评分31
隐藏状态相似度可预测接种提示后的行为复现
Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting
AI 导读
研究发现,评估提示能否重新诱发接种提示所抑制的有害行为,文本相似度并非完整预测指标;当提示诱发的激活状态与接种上下文相似时,行为更易复现。该结果推进了对接种提示如何调节条件性失配的理解,并表明激活空间分析可帮助识别被抑制行为在评估提示下是否仍可被触及。
正文 · 原文
Abstract:Fine-tuning on narrow harmful tasks can cause emergent misalignment, where models generalize harmful behavior beyond the training distribution. Inoculation prompting can reduce this effect by explicitly eliciting the undesired behavior during training, but recent work shows that the behavior can reappear when evaluation prompts contain cues from the training context. We study what makes such prompts effective triggers. We find that, surprisingly, textual similarity to the inoculation prompt is an incomplete predictor. Instead, prompts are more likely to re-elicit suppressed behavior when they induce activation states similar to those produced by the inoculation context. These findings advance our understanding of how inoculation prompting modulates conditional misalignment, and suggest that activation space analysis can help identify when suppressed behaviors remain accessible under eval-time prompts.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23096 [cs.AI] |
| (or arXiv:2611.23096v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23096 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rose Nayoung Kwon [view email]
[v1]
Tue, 17 Nov 2026 17:20:33 UTC (402 KB)
来源:arXiv cs.AI · arxiv.org