arXiv cs.AI· Sergei Katkov, Antonio Liotta·· 3 天前AI 评分22
Whisper 个性化增益研究：LoRA 微调效果并非普遍有效
Understanding Personalization Gains in Whisper
AI 导读
研究对比了零样本 Whisper、池化 LoRA 和分组 LoRA 在 Common Voice、LibriSpeech 和 VoxPopuli 上的表现，发现 LoRA 增益集中在困难的 Common Voice 分组，在 VoxPopuli 上表现一致，但在低错误率的 LibriSpeech 上基本消失。
正文 · 原文
Abstract:Whisper-style ASR models provide strong zero-shot transcription, but their residual errors are not evenly distributed across datasets, recording conditions, and metadata-based groups. We study when lightweight low-rank adaptation (LoRA) improves Whisper and whether observed gains require group-specific adapters. Across Common Voice, LibriSpeech, and VoxPopuli, we compare zero-shot Whisper, pooled LoRA, and group-specific LoRA under matched within-group splits. We find that adaptation is not uniformly beneficial: gains are concentrated in difficult Common Voice groups, are consistent on VoxPopuli, and largely disappear on low-error LibriSpeech. Crucially, pooled LoRA often matches, and sometimes clearly outperforms, group-specific LoRA. Gains over zero-shot should therefore be tested against matched pooled adaptation before being interpreted as evidence of personalization.
| Comments: | 14 pages. Presented at SPECOM 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.17019 [cs.AI] |
| (or arXiv:2610.17019v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17019 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Lecture Notes in Computer Science, vol. 16934, pp. 3-16, Springer (2027) |
| Related DOI: | https://doi.org/10.1007/978-3-032-37867-5_1
DOI(s) linking to related resources |
Submission history
From: Sergei Katkov [view email]
[v1]
Mon, 12 Oct 2026 15:59:09 UTC (47 KB)
来源：arXiv cs.AI · arxiv.org