arXiv cs.CL· Yixian Tan, Aulia Adila, Jonathan Heng, Kunat Pipatanakul, Donghang Wu, Anocha Sutaveephamochanon·· 3 天前AI 评分29
SEA-HELM Audio：评估多语言 ASR-LLM 流水线在 SEA-HELM 各子集上的表现
SEA-HELM Audio: Evaluating Multilingual ASR-LLM Pipeline Performance Across SEA-HELM Subsets
AI 导读
SEA-HELM Audio 将文本版 SEA-HELM 基准扩展到语音领域，用合成语音对比中文、英语、马来语、印尼语和泰米尔语五种语言在 ASR-LLM 级联流水线中的语音与文本输入表现。结果显示语音输入带来显著性能下降：多项选择推理准确率下降 9.00 至 18.50 个百分点，指令遵循下降 20.00 至 42.86 个百分点，泰米尔语降幅最大，超过 40 个百分点。
正文 · 原文
Abstract:We introduce SEA-HELM Audio, a Southeast Asian (SEA) multilingual evaluation framework that extends the text-based SEA-HELM benchmark to assess performance degradation in cascaded automatic speech recognition and large language model (ASR-LLM) pipelines. Using synthesized speech, we compare speech-input and text-input performance across five languages (Chinese, English, Malay, Indonesian, and Tamil) on multiple-choice reasoning and instruction-following tasks. We evaluate open-weight and commercial ASR-LLM systems using identical verifiers and quantify the speech gap as the difference between text and speech accuracy. Results show substantial degradation, with accuracy drops of 9.00 to 18.50 percentage points for multiple-choice reasoning and 20.00 to 42.86 points for instruction following. Tamil exhibits the largest degradation, exceeding 40 percentage points. Our findings highlight the limitations of text-only benchmarks in predicting voice-agent reliability and demonstrate the value of SEA-HELM Audio for evaluating multilingual voice-agent systems.
| Subjects: | Sound (cs.SD); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19869 [cs.SD] |
| (or arXiv:2610.19869v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19869 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Anocha Sutaveephamochanon PhD [view email]
[v1]
Wed, 14 Oct 2026 09:29:36 UTC (39 KB)
来源：arXiv cs.CL · arxiv.org