arXiv cs.AI· Kia-J\"ung Yang, Dominik Meier, Jiachen Zhao, Terry Ruas, Bela Gipp·· 12 小时前AI 评分34
研究揭示 CoT 如何干扰大推理模型的拒绝行为转向
Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusal
AI 导读
在 DeepSeek-R1-Distill-LLaMA-8B 上，固定 CoT 时激活转向仅能逆转 39% 的拒绝案例，而完全移除 CoT 后这一比例升至 70%，说明 CoT 会主动强化拒绝。
正文 · 原文
Abstract:Large reasoning models (LRMs) generate chain-of-thought (CoT) traces before producing final outputs, introducing a dynamic internal state that may complicate control mechanisms such as refusal. Unlike instruction-tuned LLMs, where refusal is mediated by a single directional subspace, refusal in large reasoning models (LRMs) additionally depends on the CoT. In DeepSeek-R1-Distill-LLaMA-8B, activation steering reverses refusal in only 39% of cases when the CoT is kept fixed, but removing the CoT entirely increases this to 70%, indicating that the CoT actively reinforces refusal. In a two-stage intervention where the model regenerates its CoT under activation steering, refusal is reversed in 94% of cases, while the resulting CoT alone retains 48% of this effect even after steering is removed. This suggests that the CoT can carry and reconstruct the compliance signal independently. These findings indicate that refusal in LRMs is jointly encoded in residual stream activations and CoT. This joint activation makes LRM more robust against activation-level interventions alone, but exposes CoT to a possible alternative surface attack.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.26772 [cs.AI] |
| (or arXiv:2605.26772v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.26772 arXiv-issued DOI via DataCite |
|
| Journal reference: | ICML 2026 Workshop on Mechanistic Interpretability |
Submission history
From: Kia-Jüng Yang [view email]
[v1]
Tue, 26 May 2026 09:41:15 UTC (84 KB)
[v2]
Mon, 19 Oct 2026 23:55:05 UTC (84 KB)
来源：arXiv cs.AI · arxiv.org