跳到正文
arXiv cs.AI· Youngwon Choi, Donghyuk Jung, Hyeonyu Kim, Taeyoun Kwon, Myeongkyun Cho, Moon Hwan Kim·· 10 小时前AI 评分26

仅文本适配的语音 LLM 在口语语义解析中的鲁棒性对比研究

Robustness of Text-Only Adapted Speech LLMs for Spoken Semantic Parsing

AI 导读

研究对比了 Speech LLM 与 ASR-LLM 级联在仅文本任务监督下的鲁棒性,使用 3B 和 7B 模型在三个数据集上测试干净语音及五种 SNR 的 babble 噪声。结果显示 Speech LLM 在命令理解、意图预测和函数调用上整体更优,尤其在高噪声下,但细粒度实体预测提升不稳定;其单样本平均推理时间也更短。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models (LLMs) are increasingly used to generate structured outputs from natural language instructions, including semantic frames and function calls. When the input is speech, this task becomes spoken semantic parsing, which can be performed using either Speech LLMs or ASR-LLM cascades. Although both approaches can be adapted with text-only task supervision, their relative robustness to acoustic degradation remains unclear. We compare these approaches under matched text-form supervision using 3B and 7B models across three datasets, evaluated on clean speech and babble noise at five SNR levels. Results show that Speech LLMs generally achieve better performance in command understanding, intent prediction, and function calling, particularly under severe noise, while improvements in fine-grained entity prediction are less consistent. An additional evaluation shows that Speech LLMs achieve lower average inference time per sample than ASR-LLM cascades at both model scales.
Comments: review in progress
Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Cite as: arXiv:2606.21453 [cs.HC]
  (or arXiv:2606.21453v2 [cs.HC] for this version)
  https://doi.org/10.48550/arXiv.2606.21453

arXiv-issued DOI via DataCite

Submission history

From: Youngwon Choi [view email]
[v1] Fri, 19 Jun 2026 14:11:10 UTC (179 KB)
[v2] Wed, 18 Nov 2026 13:14:43 UTC (19 KB)

来源:arXiv cs.AI · arxiv.org