arXiv cs.AI· Jeevan Neupane, Janardan Bhetwal, Kshitiz Paudel, Anup Aryal, Bipin Bashyal·· 10 小时前AI 评分22
尼泊尔语-英语混合语音识别：SraVaani 语音基础模型的数据准备与适配
Nepali-English Code-Mixed Speech Recognition: Data Preparation and Adaptation of a Speech Foundation Model
AI 导读
研究团队用 172.8 小时、覆盖六个尼泊尔语语音领域的语料适配 449M 参数的混合 TDT-CTC 模型 SraVaani，全量微调将测试 WER 从 48.6% 降至 17.1%，12 个 epoch 的运行达到 16.3%，与微调后的 Whisper large-v3-turbo 的 16.4% 相当。
正文 · 原文
Abstract:Nepali speakers regularly mix English words into conversation, but multilingual speech recognisers often struggle with language switches and the writing script used for English words. We study how to adapt SraVaani, a 449M-parameter hybrid TDT-CTC model, using a 172.8-hour corpus covering six Nepali speech domains. Its pretrained tokenizer does not support digits and several common symbols. Rather than replace it or discard these recordings, we generate Nepali and English readings for numbers and use zero-shot ASR output to select a plausible form. Full fine-tuning reduces test word error rate (WER) from 48.6% to 17.1%; a 12-epoch run reaches 16.3%, compared with 16.4% for fine-tuned Whisper large-v3-turbo. Freezing the encoder increases WER to 25.5%, while excluding numeric clips raises WER on numeric test speech from 14.3% to 21.3%. We also evaluate on FLEURS using both its original references and mixed-script alternatives. The comparison shows that transcription conventions explain part, but not all, of the out-of-domain performance loss.
| Comments: | 9 pages, 5 figures, 13 tables. Code and model checkpoints: this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2611.13512 [cs.AI] |
| (or arXiv:2611.13512v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13512 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jeevan Neupane [view email]
[v1]
Tue, 10 Nov 2026 03:39:19 UTC (284 KB)
来源：arXiv cs.AI · arxiv.org