arXiv cs.CL· Anthony Nguyen, Wenjun Lin·· 11 小时前AI 评分38
研究：上下文压缩会削弱模型的拒答对齐吗？
Does Context Compression Preserve Refusal Alignment?
AI 导读
研究发现，基于编码器的上下文压缩虽重建保真度高，却会系统性削弱指令微调语言模型的拒答行为，且该效应跨模型家族与压缩架构普遍存在。机制分析显示压缩会衰减解码器拒答方向上的激活，而推理时干预方法 Memory Steering 可在不重训练的情况下将拒答率恢复至接近基线水平。
正文 · 原文
Abstract:Context compression reduces inference cost by encoding inputs into compact representations while preserving semantic content. An open question is whether semantic preservation alone is sufficient to maintain downstream behaviours such as refusal alignment. We investigate this question and find that encoder-based compression systematically weakens refusal behaviour in instruction-tuned language models, despite high reconstruction fidelity. This effect persists across model families and compression architectures. Mechanistic analysis shows that compression attenuates activation along the decoder's learned refusal direction. We further explore Memory Steering, a lightweight inference-time intervention that restores refusal rates to near-baseline levels without retraining and operates entirely in compressed representation space. These results demonstrate that semantic preservation does not guarantee behavioural preservation under compression, highlighting the need to explicitly preserve alignment-relevant features in compression-aware systems.
| Comments: | Presented at Canadian AI 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.06380 [cs.AI] |
| (or arXiv:2611.06380v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06380 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Anthony Nguyen [view email]
[v1]
Wed, 4 Nov 2026 11:42:42 UTC (583 KB)
来源：arXiv cs.CL · arxiv.org