arXiv cs.CL· Roi Yozevitch, Or Haim Anidjar·· 12 小时前AI 评分26
Whisper 微调新方法：用嵌入向量与 k-center 算法智能挑选训练子集
Whispering Wisely: Smart Subset Selection for ASR Fine-Tuning
AI 导读
研究者提出一种基于嵌入向量的子集选择方法，利用 Whisper 预训练编码器将音频映射到固定维度空间，再用贪心 k-center 算法挑选多样且有代表性的训练数据，以降低 ASR 微调成本。在希伯来语和丹麦语上的评测显示，k-center 选择在较小数据预算下常优于随机采样，且其子集覆盖的语言内容更广。
正文
Abstract:Automatic Speech Recognition (ASR) models such as OpenAIs Whisper have substantially improved speech recognition quality but remain expensive to fine-tune, especially when large amounts of transcribed audio are available. This makes it challenging for many practitioners to adapt these models to new languages or domains. In this work, we introduce an embedding-based subset selection approach for selecting smaller training sets for ASR fine-tuning. Using Whispers pretrained encoder, we map each audio sample into a fixed-dimensional embedding space and apply a greedy k-center algorithm to select diverse and representative subsets of the training data. We evaluate this approach on both Hebrew and Danish, showing that k-center selection often improves upon random sampling, particularly at smaller subset sizes. An analysis of lexical diversity reveals that k-center subsets capture a broader range of linguistic content than random subsets of the same size, which may help explain their observed effectiveness. Overall, our results demonstrate that embedding-based subset selection provides a practical strategy for ASR subset selection in the two evaluated languages, with the clearest recognition benefits observed at smaller sample budgets.
| Comments: | 17 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32997 [cs.CL] |
| (or arXiv:2610.32997v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32997 arXiv-issued DOI via DataCite |
Submission history
From: Roi Yozevitch [view email]
[v1]
Mon, 5 Oct 2026 10:47:16 UTC (2,314 KB)
来源：arXiv cs.CL · arxiv.org