arXiv cs.CL· Qizhou Wang, Jiawen Zhu, Guansong Pang, Christopher Leckie·· 12 小时前AI 评分29
AnomalyJev：用声明级验证评估图像异常解释
AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level Verification
AI 导读
AnomalyJev 是一个轻量框架，通过从多模态 LLM 生成的异常检测解释中提取可验证声明，并与真实标注比对来评估解释的忠实度与完整性，无需 token 生成。研究构建了含 640 条解释、超 1,600 条专家评分的评测集，覆盖域内与跨数据集设置。AnomalyJev 与人类标注者的一致性超过多数开源 MLLM 评判器，接近旗舰 MLLM 评判器水平，而运行时间和资源消耗仅为其一小部分。
正文 · 原文
Abstract:Recent multimodal LLM (MLLM)-based anomaly detection (AD) methods can generate textual explanations that help interpret their predictions. However, reliably and efficiently evaluating these explanations remains challenging. While LLM judges can assess explanations through structured rubrics, they often require substantial token generation, making evaluation costly and difficult to scale. We instead propose to anchor evaluation in the available ground truth by extracting verifiable claims from each explanation and verifying them against the corresponding annotations. We instantiate this idea as AnomalyJev, a lightweight framework that extracts verifiable claims from contextualised representations using ground-truth-defined queries and pointer based heads, without token generation. The claims are then verified against the ground truth to score faithfulness and completeness. For evaluation, we construct a human-annotated set of 640 explanations with over 1,600 expert ratings across in-domain and cross-dataset settings. Despite its compact design, AnomalyJev achieves stronger agreement with human annotators than most open MLLM judges and approaches that of flagship MLLM judges, at a fraction of their runtime and resources.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.7; I.4.9; I.2.10 |
| Cite as: | arXiv:2610.27565 [cs.CV] |
| (or arXiv:2610.27565v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27565 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qizhou Wang [view email]
[v1]
Tue, 20 Oct 2026 03:29:08 UTC (832 KB)
来源：arXiv cs.CL · arxiv.org