arXiv cs.CL· Karl El Hajal, Mathew Magimai. -Doss·· 11 小时前AI 评分26
OLIVE：面向语音 SSL 的波形重建视图增强潜在预测框架
OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSL
AI 导读
OLIVE 是一个联合优化分析与合成目标的自监督语音表示学习框架，在单一编码器内实现掩码潜在预测与波形重建的功能分离。据称它是首个直接重建波形、在预训练后保留声码器且在判别任务上保持竞争力的语音 SSL 模型。加入重建基本保持了纯掩码潜在预测的语音识别性能，同时提升波形重建与说话人任务表现。
正文 · 原文
Abstract:We propose Online Latent prediction with Invariant Views and rEconstruction (OLIVE), a self-supervised speech representation learning framework that jointly optimizes analysis and synthesis objectives. OLIVE couples masked latent prediction and waveform reconstruction through a functional separation within a single encoder: reconstruction constrains early encoder features to retain signal-level information, while masked latent prediction shapes later contextual representations for robust downstream performance, with waveform view augmentations defining the invariances they learn. To the best of our knowledge, OLIVE is the first speech SSL model that directly reconstructs waveforms, retains its vocoder after pre-training, and remains competitive on discriminative tasks. Adding reconstruction largely preserves the speech recognition performance of masked latent prediction alone, while improving waveform reconstruction and speaker tasks. Adding waveform mixup and gain augmentations provides a controllable way to shift the representations further toward speaker, generation, and reconstruction tasks, while remaining competitive on recognition and semantic tasks.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2606.30356 [cs.CL] |
| (or arXiv:2606.30356v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.30356 arXiv-issued DOI via DataCite |
Submission history
From: Karl El Hajal [view email]
[v1]
Mon, 29 Jun 2026 14:24:19 UTC (1,240 KB)
[v2]
Tue, 3 Nov 2026 23:41:47 UTC (1,247 KB)
来源：arXiv cs.CL · arxiv.org