arXiv cs.AI· Sascha Saralajew, Eleonora Poeta, Eliana Pastor, Ammar Shaker·· 12 小时前AI 评分27
审计 CUB 上 Concept Bottleneck 模型的准确率来源
Auditing Where Concept Bottleneck Accuracy Comes From on CUB
AI 导读
研究审计了 CBM 与 CEM 在 CUB-200-2011 上的准确率来源，发现 CUB 标注噪声过大：类级训练标签与逐图标注仅在约 82% 的图像-概念对上一致，基于逐图标注训练的分类器物种准确率仅约 37%。
正文 · 原文
Abstract:Concept Bottleneck Models (CBMs) promise predictions that humans can inspect and correct through concepts such as "curved beak". Yet on CUB-200-2011, their standard benchmark, progress is measured like that of ordinary classifiers: by task and concept accuracy. Neither reveals whether the label is predicted from the exposed concept scores, or whether a concept is supported by visual evidence of that concept rather than by co-occurring features such as cues to the species as a whole. We audit where the accuracy of CBMs and Concept Embedding Models (CEMs) on CUB comes from. The CUB annotations are too noisy to explain it: class-level training labels agree with per-image annotations on only ~82% of image--concept pairs, and classifiers trained on per-image annotations reach only ~37% species accuracy. Leakage probes show that CEM predictions cannot be reconstructed from their concept scores; the missing class information sits in internal embeddings. We identify four design axes through which class information can enter concept scores---receptive field, shared versus concept-specific representations, norm-dependent scoring, and task supervision. An ablation shows that most of it enters through non-local features: restricting a ResNet50 backbone to local BagNet patches lowers the accuracy recoverable from concept scores from 72% to 43%, close to what ground-truth annotations support, while joint training raises it again at the cost of concept accuracy. Localization and occlusion show that locality makes grounding measurable but does not guarantee it: under class-level labels, local concept scores use the head as a proxy for body, wing, and tail concepts, and CEM concept scores barely respond to occluding their supporting part. Task and concept accuracy should thus support, not drive, the evaluation of CBMs, and given the noisy annotations, CUB accuracy gaps should not be overinterpreted.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.28059 [cs.CV] |
| (or arXiv:2610.28059v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28059 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sascha Saralajew [view email]
[v1]
Tue, 20 Oct 2026 11:47:49 UTC (2,639 KB)
来源：arXiv cs.AI · arxiv.org