arXiv cs.CL· Koki Ryu, Hitomi Yanaka·· 12 小时前AI 评分38
VocalGrad：评估音频语言模型的声学感知能力
VocalGrad: Evaluating Acoustic Perception in Audio Language Models
AI 导读
研究者提出 VocalGrad 基准，将声学感知评估简化为时间变化方向的二分类任务，以剥离高层推理与语义理解的影响。结果显示，人类标注者准确率很高，而当前最先进的音频语言模型（LALMs）仅接近随机水平。线性探测进一步揭示两类失败模式：部分类别信息保留在内部表示中但未被语言模型头利用，另一些类别信息随层数加深逐渐难以线性解码。
正文 · 原文
Abstract:Recent large audio language models (LALMs) have extended their capabilities beyond lexical speech understanding to tasks requiring paralinguistic and non-linguistic audio understanding, such as emotion recognition. However, these models still struggle to reliably utilize acoustic information, particularly when it co-occurs with spoken linguistic content. Existing benchmarks do not clearly distinguish acoustic perception from higher-level reasoning, making it difficult to identify the source of these failures. To address this issue, we introduce VocalGrad, a benchmark designed to evaluate LALMs' ability to perceive low-level acoustic variations. VocalGrad formulates this problem as a simple binary classification of temporal change direction, enabling evaluation largely independent of higher-level reasoning or semantic understanding. Our benchmark results reveal a substantial gap in acoustic perception between human annotators and state-of-the-art LALMs: while human annotators achieve high accuracy, LALMs perform at near-chance levels. We further conduct linear probing to identify structural bottlenecks in these models. Our analysis reveals two distinct failure modes: for some categories, task-relevant information is preserved in internal representations but not utilized by the language model head; for others, the information becomes progressively less linearly decodable across model layers. Overall, our results reveal substantial limitations of LALMs in modeling acoustic variation under controlled settings and expose representation differences not observable from text outputs.
| Comments: | NeurIPS 2026 (Evaluations & Datasets Track). Dataset page: this https URL |
| Subjects: | Sound (cs.SD); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23745 [cs.SD] |
| (or arXiv:2610.23745v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23745 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Koki Ryu [view email]
[v1]
Fri, 16 Oct 2026 16:54:35 UTC (1,840 KB)
来源：arXiv cs.CL · arxiv.org