arXiv cs.CL· Anglin Cai, Zhiqi Wang, Xiaozhi Yang, Yurun Lin, Dongwon Lee, Yuchen Yang·· 12 小时前AI 评分52
看似合理的解释足以误导多模态模型裁判
A Plausible Explanation Is Enough to Mislead Multimodal Judges
AI 导读
研究提出 JUPE 基准，在视觉问答任务中固定答案、只改变解释，发现 14 个多模态大模型裁判会被听起来合理的普通解释误导，效果平均上与刻意说服性论证相当。普通理由和思维链理由分别使错误答案的被拒率下降 12.7 和 13.8 个百分点，作者称之为合理性俘获。信号检测分析显示判定标准显著向接受一侧偏移，而辨别力无明显变化，现有缓解措施保护有限。
正文 · 原文
Abstract:Multimodal large language models (MLLMs) increasingly serve as judges in training and evaluation pipelines, often assessing answers together with explanations. Although AI judges are sensitive to textual factors, it remains unclear whether ordinary plausible explanations can sway judgments requiring visual verification. We introduce JUPE, a visual-question-answering (VQA)-style benchmark that fixes answers while varying explanations.
Across 14 MLLM judges, ordinary explanations that sound plausible mislead judges, with effects comparable on average to deliberately persuasive arguments. Plain justifications and chain-of-thought rationales reduce rejection of incorrect answers by 12.7 and 13.8 percentage points, respectively. We call this failure plausibility capture. Using signal detection theory, we find a significant mean shift in decision criterion toward acceptance, without a clear change in mean discriminability. Explanations sway judgments across domains, explanation generators, and judging protocols, while tested mitigations offer limited, model-dependent protection. Such shifts can inflate performance estimates when judges evaluate answers with explanations.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.22631 [cs.CL] |
| (or arXiv:2610.22631v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22631 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Anglin Cai [view email]
[v1]
Thu, 15 Oct 2026 20:38:53 UTC (1,277 KB)
来源：arXiv cs.CL · arxiv.org