arXiv cs.CL· Zhaolu Kang, Yidi Wang, Yile Li, Siqi Zeng, Yingjie He, Richeng Xuan, Zhichao hu·· 13 小时前AI 评分36
TRIAD：多模态大模型全模态歧义消解基准
TRIAD: Benchmarking Omni-Modal Ambiguity in Multimodal Large Language Models
AI 导读
TRIAD 是面向全模态歧义消解的新基准，包含 327 条人工编写查询、86 个场景组和 18 类歧义，每条为文本-图像-音频三元组，需三模态齐备才能唯一确定答案。评测 14 个全模态系统显示，最佳模型组准确率仍落后人类超 20 分；移除选项后模型组准确率降至个位数，人类依然稳健。留一法与转写/描述反事实实验表明，当前系统在文本与图像存在时往往未充分利用音频线索。
正文 · 原文
Abstract:Ambiguity is a central feature of natural communication: a sentence, image, or sound may remain underspecified until evidence from the other channels is considered. Yet current multimodal benchmarks mostly test perception, recognition, or reasoning over already determinate inputs, leaving unclear whether omni-modal large language models can resolve ambiguity distributed across text, vision, and audio. We introduce TRIAD, a benchmark for omni-modal ambiguity resolution. Each item is a text--image--audio triplet paired with a question and a set of answer options, and is constructed so that the full triplet determines a unique answer while removing any one modality makes the set of consistent options non-unique. TRIAD contains $327$ hand-written queries in $86$ scene groups and covers $18$ ambiguity categories across the three modalities. Evaluating $14$ omni-modal systems reveals that strong item-level performance does not translate into robust scene-level disambiguation: the best model still trails humans by over $20$ points in group accuracy. Removing answer options widens the gap further, driving model group accuracy to the single digits while humans remain robust. Leave-one-out and transcript/description counterfactuals further show that current systems often underuse audio cues when text and image are present. TRIAD reframes omni-modal evaluation around disambiguation rather than recognition, and provides a taxonomy, benchmark, and diagnostic protocol for measuring this capability.
| Comments: | NeurIPS 2026 Evaluations and Datasets Track |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.30841 [cs.CL] |
| (or arXiv:2610.30841v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30841 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yingjie He [view email]
[v1]
Thu, 22 Oct 2026 07:15:40 UTC (1,374 KB)
来源：arXiv cs.CL · arxiv.org