arXiv cs.AI· Vasundra Srinivasan·· 2 天前AI 评分18
论文提出多模态一致性评分 MCS：高分不等于好数据
Good Scores, Bad Data: A Metric for Multimodal Coherence
AI 导读
研究者提出多模态一致性评分（MCS），一种不依赖下游模型、无需人工标注即可评估多模态融合质量的指标，将一致性拆分为 identity、spatial、semantic、decision 四个维度并用 Nelder-Mead 优化学习权重。
正文 · 原文
Abstract:Multimodal AI systems are evaluated by downstream task accuracy, but high accuracy does not mean the underlying data is coherent. A model can score well on Visual Question Answering (VQA) while its inputs contradict each other. We introduce the Multimodal Coherence Score (MCS), a metric that evaluates fusion quality independent of any downstream model. MCS decomposes coherence into four dimensions, identity, spatial, semantic, and decision, with weights learned via Nelder-Mead optimization. We evaluate on 1,000 Visual Genome images using DETR, CLIP, and ViLT, and validate on 150 COCO images with no retraining. Across three fusion architectures, MCS discriminates quality with higher sensitivity than task accuracy alone (Spearman rho = 0.093 vs. 0.071). Perturbation experiments confirm each dimension responds independently to its failure mode with zero cross-talk. MCS is lightweight, requires no human annotation, and tells you not just that something broke, but what broke.
| Comments: | 9 pages, 6 figures, NeurIPS 2024 format |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2603.25924 [cs.CV] |
| (or arXiv:2603.25924v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2603.25924 arXiv-issued DOI via DataCite |
Submission history
From: Vasundra Srinivasan [view email]
[v1]
Thu, 26 Mar 2026 21:30:34 UTC (2,259 KB)
[v2]
Sat, 24 Oct 2026 00:43:12 UTC (2,259 KB)
来源：arXiv cs.AI · arxiv.org