arXiv cs.CL· Joshua Muhumuza, Cornelius Kalenzi·· 2 天前AI 评分41
ViT 中的虚假注意力头：医学胸部 X 光分析中标注驱动偏见的机制识别
Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis
AI 导读
研究用机制可解释性在 ViT-Base 16 中定位到一个专门读取胸部 X 光片角落标注文字的注意力头 L8H10，在 200 张 COVID-19 图像中有 62% 关注标注而非肺部病变。
正文 · 原文
Abstract:Vision Transformers deployed in medical imaging exploit spurious correlations, yet the internal mechanisms remain unidentified and impossible to fix. We apply mechanistic interpretability to locate one such mechanism in ViT-Base 16: a single attention head (L8H10) that functions as a dedicated annotation-reader on chest X-rays, attending to corner text rather than lung pathology in 62% of 200 COVID-19 images. Causal validation via inpainting confirms annotation content rather than luminance or edge density drives the behaviour, increasing lung attention 1.52 times upon removal in affected images ($p{<}0.0001$). The finding replicates across NIH ChestX-ray14 (79%), DeiT-Base (88%), and ViT-Large 16 (80%), establishing spurious annotation-attending heads as a consistent property of ViT-family architectures across two datasets and three architectures. Full fine-tuning does not suppress the behaviour and may entrench it: 98.4% of spurious images remain spurious and the spurious rate rises marginally, as the shortcut is reinforced rather than removed. Critically, single-head ablation produces $p{=}0.340$ and multi-head ablation of all three identified carriers produces $p{=}0.532$, meaning a causally verified annotation-reading circuit is completely invisible to output-level evaluation. Mechanistic auditing finds what behavioural testing cannot.
| Comments: | 8 pages, 8 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21638 [cs.CV] |
| (or arXiv:2610.21638v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21638 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Joshua Muhumuza Mr [view email]
[v1]
Thu, 15 Oct 2026 10:21:14 UTC (926 KB)
来源：arXiv cs.CL · arxiv.org