arXiv cs.CL· Basavaraj R M, Aryan Bharat Kumar·· 10 小时前AI 评分24
注意力手术的合理性检查：表示驱动设计中的零模型与匹配干预
Sanity Checks for Attention Surgery: Null Models and Matched Interventions for Representation-Motivated Design
AI 导读
研究提出三项可复用检查，用于验证基于内部表示统计量的 Transformer 架构修改是否成立，并以自注意力移除为案例。在九个预训练模型（5,408 个注意力头，124M 至 6.9B 参数）上，早期位置因果零模型可解释头-值余弦对齐的 48% 至 63%。
正文 · 原文
Abstract:Transformer architectural modifications frequently measure a statistic on internal representations, observe that it is large, remove it architecturally, and report an improvement. Interpreting that sequence requires evaluating whether the statistic exceeds its noise floor, whether it survives a geometric null, and whether the training gain requires the specific direction removed. We formulate these requirements as three reusable checks and apply them to self-attention removal, which deletes the component of an attention head's output aligned with its own value vector. Across nine multi-head pretrained models (5,408 heads, 124M to 6.9B parameters), an earlier-position causal null accounts for 48% to 63% of the head-value cosine alignment, with no monotone scale trend. Under grouped-query attention, this alignment is specific to each head's own key-value group and reverses across groups (+0.24 to +0.27 within-group, -0.11 to -0.19 across-group). Split-half reliability demonstrates that the per-head intervention effect is resolvable in GPT-2 ($r_\Delta = 0.80$) but attenuated in Pythia-160M (0.42) and Pythia-410M (0.53), whereas structural self-cosine reliability is near-perfect in all three ($r_{\text{stat}} > 0.99$); which statistic predicts its own intervention is model-dependent. In a pre-registered 3-arm by 8-seed paired factorial at $4.0 \times 10^8$ tokens per run (399,900,672 tokens), a fixed arbitrary rank-one direction does not differ from baseline (+0.0011 nats, $p = 0.197$), while self-directed removal lowers validation loss (-0.0029 nats, Holm-corrected $p = 0.0023$) and outperforms the fixed arbitrary control directly (-0.0040 nats, $p = 0.0018$). At 50.9M scale the training benefit requires the own-value direction rather than a fixed arbitrary one, but a large cosine alignment alone does not establish redundancy in frozen models.
| Comments: | 13 pages, 7 figures, 4 tables. Code: this https URL (archived at this https URL) |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.13543 [cs.LG] |
| (or arXiv:2611.13543v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13543 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Basavaraj R M [view email]
[v1]
Tue, 10 Nov 2026 04:06:26 UTC (241 KB)
来源：arXiv cs.CL · arxiv.org