arXiv cs.CL· Ege Kaya (University of British Columbia)·· 9 小时前AI 评分38
4-Bit 量化对聊天模型安全行为的影响:用匹配扰动归因行为偏移
Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations
AI 导读
一项预注册研究对三个 3-4B 聊天模型做配对测试与盲人工标注,发现 4-bit 量化带来方向相反的安全行为偏移:预填合规回复时 Qwen2.5-3B 更常续写有害内容,Llama-3.2-3B 在留出越狱集上合规率低 10.3 个百分点。
正文 · 原文
Authors:Ege Kaya (University of British Columbia)
Abstract:Chat models on phones and laptops are usually 4-bit copies of the model that was evaluated, and audits of such copies disagree about what quantization does to refusal. A 4-bit build is one of many weight perturbations of the same size, so a change in its behaviour could come from its specific rounding error or from any perturbation that large. In a pre-registered study of three 3-4B chat models, with paired tests, blind human labels and an 8-bit null, we find two 4-bit shifts of opposite sign: with a compliant reply prefilled, Qwen2.5-3B continues into harmful content more often, and on held-out jailbreaks Llama-3.2-3B complies 10.3 points less often. Matched perturbations separate the two. Shuffled and Gaussian versions of the error average close to zero, while scaling the real error, from its mirror image to twice its size, changes behaviour near-linearly. At half the real error, Llama's likelihood is almost unchanged, yet it already complies 5.8 points less often, and this http URL's Q4_K_M, at the same bits per weight, moves neither model. Refusal under 4-bit quantization depends on the specific rounding error, not on bits per weight or likelihood, so each build has to be compared with its original.
| Comments: | 16 pages, 4 figures. Code and labels: this https URL |
| Subjects: | Cryptography and Security (cs.CR); Computation and Language (cs.CL) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2611.22186 [cs.CR] |
| (or arXiv:2611.22186v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22186 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ege Kaya [view email]
[v1]
Tue, 17 Nov 2026 04:20:17 UTC (139 KB)
来源:arXiv cs.CL · arxiv.org