arXiv cs.CL· Maryem Bouziane, Salima Mdhaffar, Yannick Est\`eve·· 2 天前AI 评分22
通过多教师蒸馏学习语音属性表示
Learning Speech Attribute Representations via Multi-Teacher Distillation
AI 导读
研究提出一种多属性多教师蒸馏框架，用共享 SSL 骨干网络和属性专属分支，从四个冻结教师模型学习语义、说话人、语言和情感表示。在多语言语义检索、说话人验证、语言识别和语音情感识别任务上，该模型性能接近专用单任务学生模型，且仅需一个共享编码器。学习到的层聚合策略还揭示不同语音属性自然涌现于 SSL 编码器的不同深度。
正文 · 原文
Abstract:Recent post-training approaches have shown that self-supervised speech encoders can be adapted to learn a single utterance-level speech attribute through knowledge distillation. In this work, we investigate whether a single speech encoder can simultaneously preserve multiple complementary utterance-level speech representations. We propose a multi-attribute multi-teacher distillation framework that learns semantic, speaker, language, and emotion representations from four frozen teacher models using a shared SSL backbone and attribute-specific branches. Experiments on multilingual semantic retrieval, speaker verification, language identification, and speech emotion recognition show that the proposed model achieves performance close to dedicated single-task students while requiring only a single shared encoder. Furthermore, the learned layer aggregation strategies reveal that different speech attributes naturally emerge at different depths of the SSL encoder, providing new insights into the organization of speech representations.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21941 [cs.CL] |
| (or arXiv:2610.21941v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21941 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yannick Estève [view email]
[v1]
Thu, 15 Oct 2026 14:15:32 UTC (3,703 KB)
来源：arXiv cs.CL · arxiv.org