arXiv cs.CL· Qing Wang, Qihan Hu, Hang Lv, Yan Yang, Boqing Liu, Xuechao Zhu, Zehan Li, Hongjie Chen, Jian Kang, Jie Li, Yongxiang Li, Lei Xie·· 11 小时前AI 评分34
从转录到重构：GSR-Bench 如何定义并评测有据生成式语音识别
From Transcription to Reformulation: Defining and Benchmarking Grounded Generative Speech Recognition
AI 导读
研究者提出 Grounded Generative Speech Recognition（GSR），将语音识别重新定义为生成有语义依据的文本而非单一转录，并建立包含字面转录、规范化重构、语义重构和语音有据结构化四级的能力层级。
正文 · 原文
Authors:Qing Wang, Qihan Hu, Hang Lv, Yan Yang, Boqing Liu, Xuechao Zhu, Zehan Li, Hongjie Chen, Jian Kang, Jie Li, Yongxiang Li, Lei Xie
Abstract:Speech language models have expanded automatic speech recognition (ASR) from transcription to general speech-to-text generation. Yet existing benchmarks still assume a single ground-truth transcript per utterance, an assumption that struggles when models produce readable, formatted, or structured outputs. In this work, we introduce Grounded Generative Speech Recognition (GSR), reformulating speech recognition as generating semantically grounded text instead of a single transcript. We define a grounded reformulation space that characterizes all valid textual realizations supported by the input speech, and establish a four-level hierarchy of reformulation capabilities, including literal transcription, canonical reformulation, semantic reformulation and speech-grounded structuring. Based on this formulation, we present GSR-Bench, the first comprehensive benchmark designed to evaluate generative speech recognition systems beyond exact transcript matching. GSR-Bench covers diverse scenarios including refined meeting transcription, cross-dialect translation, context-aware speech understanding, and robustness under silent or noisy acoustic conditions. To address the limitations of word-level evaluation metrics, we develop a multi-dimensional evaluation framework combining speech grounding assessment and LLM-based semantic judgment, whose reliability is validated through human preference studies. Extensive experiments across representative architectures, ranging from conventional ASR systems to end-to-end Speech-LLMs, reveal fundamental trade-offs between transcription fidelity and generative reformulation ability. Our analysis further identifies critical failure modes, including acoustic hallucination, unsupported generation, and instruction drift, providing insights into the design and evaluation of future grounded generative speech systems.
| Subjects: | Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2610.41292 [cs.CL] |
| (or arXiv:2610.41292v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41292 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qing Wang [view email]
[v1]
Fri, 30 Oct 2026 05:53:56 UTC (1,507 KB)
来源：arXiv cs.CL · arxiv.org