arXiv cs.CL· Zhiheng Qian, Junhao Du, Jing Peng, Hanqi Li, Kunyang Peng, Feng Xue, Lu Chen, Shuai Wang, Kai Yu·· 13 小时前AI 评分29
面向多语言 ASR 的语言感知智能体数据采样框架
Agentic Data Sampling Framework for Multilingual ASR via Linguistic-Aware Error-Driven Optimization
AI 导读
研究者提出一种语言感知的智能体数据采样框架，将数据采样 LLM 智能体引入多语言 ASR 训练循环，依据音系、形态、句法三个层面的错误分析评分，结合训练统计、预算约束与算法配置，在领域级、样本级和 token 级选择采样策略。在 12 种 Common Voice 语言上以 Whisper-medium 为 ASR 骨干的实验显示，该方法用更少训练数据和更低算力即超过全量数据训练，并显著加快收敛。
正文 · 原文
Abstract:Multilingual automatic speech recognition (ASR) has achieved rapid progress with large-scale pretrained models and multilingual speech corpora. Previous studies have proved the pivotal role of data sampling in multilingual ASR performance. However, existing sampling strategies largely rely on time-consuming manual, heuristic-based designs. To address this, we propose a linguistic-aware agentic data sampling framework that introduces a data sampling LLM agent into the multilingual ASR training loop. The agent is driven by a linguistically grounded rubric that analyzes ASR errors from phonological, morphological, and syntactic perspectives, and uses the resulting feedback together with training statistics, budget constraints, and algorithm profiles to select domain-level, sample-level, and token-level sampling strategies. This design turns evaluation results into adaptive data-selection decisions rather than treating WER/CER as final endpoints only. We conduct experiments on twelve Common Voice languages using Whisper-medium as the ASR backbone. The results show that our method achieves performance better than full-data training while using less training data and lower computational resources, and substantially accelerates training convergence compared to conventional sampling strategies.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.36538 [cs.CL] |
| (or arXiv:2610.36538v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36538 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhiheng Qian [view email]
[v1]
Tue, 27 Oct 2026 02:49:36 UTC (529 KB)
来源：arXiv cs.CL · arxiv.org