arXiv cs.CL· Shreeram Suresh Chandra, Ashwyn Sharma, Liang Lu, Xiutian Zhao, Leda Sari, Bj\"orn Schuller, Berrak Sisman·· 12 小时前AI 评分29
StyleInstructCaps：音频语言模型可控说话风格描述基准发布
A Benchmark for Controllable Speaking-Style Captioning in Audio-Language Models
AI 导读
研究者提出 StyleInstructCaps，首个面向音频语言模型的可控说话风格描述（SSC）标准化基准，将 SSC 形式化为指令遵循的音频-语言任务，并提供数据集与评测框架，衡量元数据接地性、幻觉、指令遵循、对未见指令与音频数据集的泛化及说话人风格一致性。
正文
Abstract:Speaking-style captioning (SSC) aims to generate natural language descriptions of how speech is delivered, capturing paralinguistic attributes such as vocal timbre, prosody, and expressivity. Many downstream applications, including conversational AI agents, controllable speech generation, and large-scale audio annotation, require dimension-specific style captions that describe targeted aspects of speech (e.g., speaker traits, emotion, or delivery style), rather than a single undifferentiated description. However, existing work lacks a unified formulation for controllable speaking-style captioning. We address this gap by formulating SSC as an instruction-following audio-language task, where explicit instructions specify the speaking-style dimensions to be described. We introduce StyleInstructCaps, the first standardized benchmark for controllable speaking-style captioning in audio-language models. StyleInstructCaps provides a dataset and evaluation framework that measures metadata groundedness, hallucination, instruction-following ability, generalization to unseen instructions and audio datasets, and speaker-style consistency. Across nine state-of-the-art audio-language models, we find that strong instruction-following ability does not necessarily imply grounded speaking-style understanding, and that SSC-specific instruction-diverse training substantially improves groundedness, controllability, and speaker consistency.
| Comments: | Accepted at AACL 2026. Code, data, and benchmark are available at this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2610.33036 [cs.CL] |
| (or arXiv:2610.33036v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33036 arXiv-issued DOI via DataCite |
Submission history
From: Shreeram Suresh Chandra [view email]
[v1]
Thu, 15 Oct 2026 00:17:09 UTC (4,125 KB)
来源：arXiv cs.CL · arxiv.org