arXiv cs.CL· Zewen Zhan, Hui Lu, Xueyuan Chen, Huimeng Wang, Yikai Huang, Xixin Wu, Shiyin Kang, Bin Xia, Zhiyong Wu·· 12 小时前AI 评分31
LibriSpeech-Real-QA：面向流式语音问答的基准测试
LibriSpeech-Real-QA: A Benchmark for Streaming Question Answering
AI 导读
研究者提出 LibriSpeech-Real-QA，一个基于 LibriSpeech 构建的实时语音交互数据集与基准，用于评估语音大模型能否在音频持续流入时识别目标词并作答。测试显示，多款主流实时商用模型仍倾向于等整段语音结束后才响应。研究还探索了多种建模设计，使预训练语音大模型能够完成该任务，并为后续研究建立了基线。
正文 · 原文
Abstract:Most speech large language models (SLMs) still process speech in an offline manner. They only start responding after they receive the complete speech segment. However, many real-world applications require streaming speech interaction, for which the audio continuously flows in and the machine is required to answer questions based on the so far received audio chunks. For example, a user may ask the system to continuously count how many times a person's name has been mentioned while they are speaking. We observe a lack of benchmarks and evaluation protocols for real-time speech interaction. Motivated by this, we propose LibriSpeech-Real-QA, a dataset and a benchmark for real-time speech interaction. We construct different types of real-time question-answer pairs from LibriSpeech to evaluate whether speech LLMs can identify target words in streaming audio. We evaluate several popular real-time commercial models and find that they still tend to respond only after the speech turn has finished. We demonstrate that we can enable a pretrained SLM to perform this task, where we explore several modeling designs and establish a solid baseline for future research.
| Comments: | 5 pages, 1 figure, 6 tables. Submitted to IEEE ICASSP 2027 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.06496 [cs.CL] |
| (or arXiv:2611.06496v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06496 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zewen Zhan [view email]
[v1]
Wed, 4 Nov 2026 13:30:54 UTC (97 KB)
来源：arXiv cs.CL · arxiv.org