arXiv cs.CL· Yulin Sun, Kele Xu, Yong Dou·· 9 天前AI 评分22
ICAP-Gate：为大音频语言模型提供机制引导的选择性听觉控制
Selective Listening: Mechanism-Guided Control of Audio Influence in Large Audio-Language Models
AI 导读
针对任务无关音频会干扰大音频语言模型文本推理的问题，研究者提出 ICAP-Gate，对每个模型架构特定的晚期音频通路施加机制引导、任务条件化的控制。在 4 个 LALM、2 个推理基准和 16 组全量评估中，ICAP-Gate 的影响率与答案翻转率点估计均低于无门控推理；固定抑制会损害全部 4 个模型的 ASR 表现，而 ICAP-Gate 在显式音频需求指令下可保持与无门控相当的 ASR 性能。
正文 · 原文
Abstract:Large audio-language models (LALMs) exploit multimodal evidence, yet task-irrelevant audio can alter text-reasoning decisions when listening is unnecessary. Aggregate Accuracy can hide this paired drift because audio-induced repairs and damages may cancel. Paired drift analysis and targeted interventions identify architecture-specific, intervention-sensitive late audio pathways as actionable control points. We introduce ICAP-Gate, which applies mechanism-guided, task-conditioned control to each model's pathway. Across four LALMs, two reasoning benchmarks, and environmental-sound and natural-speech interference, ICAP-Gate has lower point estimates for Influence Rate and Answer Flip than ungated inference in all 16 full-split model--condition evaluations. Fixed suppression degrades automatic speech recognition (ASR) across all four models, whereas ICAP-Gate matches ungated ASR performance by preserving the pathway for explicit audio-demand instructions. ICAP-Gate has lower paired-drift point estimates than mitigation prompting in all four evaluated settings and provides competitive stabilization relative to eight-sample Self-Consistency while using one generation per query; in controlled ARC measurements, Self-Consistency incurs $7.0$--$9.2\times$ ungated latency. These results establish selective modality influence control as a design principle for robust multimodal reasoning.
| Comments: | 24 pages, 5 figures, 20 tables |
| Subjects: | Sound (cs.SD); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.11196 [cs.SD] |
| (or arXiv:2610.11196v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11196 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yulin Sun [view email]
[v1]
Thu, 8 Oct 2026 04:00:47 UTC (2,253 KB)
来源：arXiv cs.CL · arxiv.org