DIAL-S2ST：为同声传译语音模型学习推理时延迟控制

DIAL-S2ST: Learning Latency Control for Simultaneous Speech-to-Speech Translation

AI 导读

DIAL-S2ST 是一种面向同声语音翻译模型的训练后方法，通过从 lookahead 预测自蒸馏复用预训练翻译能力，无需配对目标数据或奖励计算即可监督更早输出。该方法跨规模联合训练单个 LoRA 适配器并配合匹配的 lookahead 引导，使延迟可在推理时调节。在 Hibiki 上的法译英实验显示，仅 10 小时源语音即可完成适配，实现质量与延迟的灵活权衡。

正文 · 原文

View PDF HTML (experimental)

Abstract:We propose DIAL-S2ST, a post-training method for learning inference-time latency control in simultaneous speech-to-speech translation models. Our approach reuses pretrained translation capabilities through self-distillation from lookahead predictions, while limiting the influence of large text distillation losses to account for the imitation gap. This provides supervision for earlier emission without paired target data or reward computation. To make the learned timing adjustable at inference, we jointly train a single LoRA adapter across scales with matched lookahead guidance. Experiments on French-to-English translation with Hibiki demonstrate that only 10 hours of source speech enable this adaptation, providing flexible latency control with a favorable quality-latency trade-off.

Subjects:	Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Cite as:	arXiv:2610.32214 [cs.CL]
	(or arXiv:2610.32214v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.32214 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jeongsoo Choi [view email]
[v1] Fri, 23 Oct 2026 06:10:18 UTC (82 KB)

来源：arXiv cs.CL · arxiv.org