arXiv cs.CL· Wenbin Wang, Yuheng Li, Koichiro Yoshino·· 12 小时前AI 评分32
SAC-Bench：音频语言模型在语音与声学证据冲突下的基准测试
One Waveform, Two Stories: Benchmarking Speech-Acoustic Conflict in Audio-Language Models
AI 导读
研究者推出 SAC-Bench，用 616,031 段来自 188 个声音事件类别的音频片段，测试音频语言模型在语音陈述与声学证据矛盾时的表现。在五个开放权重 ALM 上，纯音频准确率往往很高，但冲突条件下 Event Identity 准确率下降 24.8–46.5 个百分点，时间顺序错误即使语音完整描述事件序列仍会出现。
正文 · 原文
Abstract:Audio-language models (ALMs) are evaluated on their understanding of both speech and environmental sounds. What remains underexplored is how they respond when spoken claims contradict acoustic evidence within the same recording. We introduce SAC-Bench, a benchmark of intra-audio speech--acoustic conflict comprising 616,031 unique clips from 188 sound-event classes across seven semantic, temporal, and spatial tasks. It pairs environmental recordings with compatible or contradictory synthesized spoken claims and includes audio-only controls where applicable. Across five open-weight ALMs evaluated on semantic and temporal tasks, high audio-only accuracy often coexists with large losses under conflicting claims; Event Identity accuracy is 24.8--46.5 percentage points lower in conflict than compatible conditions. Temporal Order failures persist even when speech follows the complete event sequence, so simultaneous speech--event masking cannot fully explain them. A spatial case study with Spatial-Omni finds lower motion accuracy associated with both spoken claims and the speech source's physical trajectory. Prompt interventions sometimes restore acoustic accuracy, but can also reduce claim-aligned answers without improving grounding or worsen performance. SAC-Bench thus separates acoustic competence, conflict robustness, and intervention effects.
| Comments: | 26 pages, 7 figures, 5 tables |
| Subjects: | Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2611.00325 [cs.SD] |
| (or arXiv:2611.00325v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00325 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenbin Wang [view email]
[v1]
Fri, 30 Oct 2026 19:47:54 UTC (2,838 KB)
来源：arXiv cs.CL · arxiv.org