arXiv cs.CL· Hongnian Huang, Hongliang Liu·· 11 小时前AI 评分20
当每个选择都正确：语言模型数值偏好的因果研究
When Every Choice Is Correct: A Causal Study of Numerical Preference in Language Models
AI 导读
研究以 Qwen3-4B 为案例，通过扰动探测与梯度归因定位神经元并做激活干预，发现一次答案前编辑可把 17 改为 15，而两次运行都先输出 1。替换单个注意力头或其余注意力头的输出都能恢复 17，但对 7 与 5 的最终 log-odds 影响不同；将后续 MLP 输出固定在编辑运行的值上，该效应差距从 0.119 nat 降至 0.0025 nat。
正文 · 原文
Abstract:When a language model chooses a number between 1 and 30, every integer in the range is valid, yet the model can favor particular answers. We use numerical choice to study how preference is processed across layers and tokens. Perturbation probing and gradient attribution nominate neurons for activation interventions. In a selected Qwen3-4B case, a pre-answer edit changes 17 to 15 while both runs first emit 1. After the edit ends, selective cache and attention-output replacements trace part of its continuing influence on the second digit. Replacing one attention head's output or the other heads' outputs jointly produces similar immediate readout contributions; both interventions restore 17 but have different final log-odds effects on 7 versus 5. Holding later multilayer perceptron (MLP) outputs at their values from the edited run without either replacement reduces this effect gap from 0.119 to 0.0025 nat. Later computation therefore changes how similar local contributions affect the continuation. Retrospective estimates across eighteen contexts capture much of the probability movement but miss the selected answer switch. Prompt tests also separate influence from answer changes: one held-out prompt retains influence without switching, and only one of twelve fresh wordings qualifies for the reader/MLP test; there, joint MLP clamping again reduces the gap between the two reader effects while the answer remains 17. These results use numerical choice as a controlled case for studying persistent state and conditional downstream processing; they do not establish the origin or a complete circuit of natural numerical preference.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.06025 [cs.CL] |
| (or arXiv:2611.06025v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06025 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hongliang Liu [view email]
[v1]
Wed, 4 Nov 2026 06:28:57 UTC (57 KB)
来源：arXiv cs.CL · arxiv.org