arXiv cs.AI· Gexin Huang, Haisheng Fu, Yanting Yang, Chen Zhao, Ruinan Jin, Jiaqi Liu, Xiaoxiao Li·· 15 小时前AI 评分31
VLMProbe：开源 VLM 探测与解释库
VLMProbe: An Open-source Library for VLM Probing and Interpretation
AI 导读
vlmprobe 是一个开源库，在统一模型接口内整合 lens 读取、注意力归因、因果干预和稀疏自编码器（SAE）分析，并提供共享适配器、校准与 checkpoint 工具、评估流程和可视化。
正文 · 原文
Abstract:Understanding how vision-language models use visual information requires connecting internal representations to model behavior. We present vlmprobe, an open-source library that integrates lens readouts, attention attribution, causal interventions, and sparse autoencoder (SAE) analysis within a common model interface. Shared adapters, calibration and checkpoint utilities, evaluation pipelines, and visualizations support systematic analysis across architectures. Using this framework, we compare methods across vision-language, language, and vision models through matched case studies and controlled evaluations of pretrained sparse dictionaries. The comparisons show that representation readouts and spatial explanations depend on the analysis objective and observation site, and that local attribution estimates can disagree with measured intervention effects. SAE evaluations further expose distinct trade-offs in reconstruction fidelity, concept readout, and spatial localization. Together, these results support combining multiple forms of evidence to evaluate an interpretation. vlmprobe provides a reusable basis for selecting methods, comparing their conclusions, and testing hypotheses about model behavior. The code is available on this http URL
| Comments: | 43 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33072 [cs.CV] |
| (or arXiv:2610.33072v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33072 arXiv-issued DOI via DataCite |
Submission history
From: Gexin Huang [view email]
[v1]
Wed, 21 Oct 2026 02:36:26 UTC (17,838 KB)
来源：arXiv cs.AI · arxiv.org