arXiv cs.CL· Guoqiang Hu, Lichao Zhang, Yizhou Peng, Yuhao Luo, Haifeng Hu, Sallen Sun, Will Yang, Eng Siong Chng·· 13 小时前AI 评分31
把奖励积分掉，而不是采样它：全双工语音模型打断与附和行为的窗口似然方法
Integrate the Reward, Don't Sample It: Window Likelihood for Interruptions and Backchannels in Full-Duplex Speech Models
AI 导读
针对全双工语音对话模型在用户打断与附和时的让出/保持决策，研究提出窗口似然（WL）方法：每个窗口采样续写，但对停止决策做闭式积分，用 noisy-OR 建模让出、生存乘积建模保持，避免二值窗口奖励的零梯度陷阱与信用泄漏。
正文 · 原文
Abstract:A full-duplex spoken dialogue model must yield when the user cuts in and hold when the user merely backchannels. Sequence-level RL with a timing reward (GRPO) is the usual recipe; we test its whole-rollout form on four open models and find two failures. A binary window reward gives zero advantage on exactly the windows that most need learning (a zero-gradient trap), and the one score per rollout is spread over listening-state decisions the reward is not written in terms of (credit leakage), so the model talks into silence or stops answering. Both come from sampling a reward that is a known function of a few binary control decisions. We propose \emph{window likelihood} (\WL): sample a continuation per window, but integrate in closed form over the stop decisions it visits and maximise the log-probability of acting correctly---a noisy-OR for yielding, a survival product for holding. On HumDial-FDBench, \WL\ gives the best yield/hold F1 on all six structurally different open models we post-train, exceeds Kyutai's released GRPO checkpoints (trained on different data), and adds no re-opening of the floor on any model.
| Subjects: | Audio and Speech Processing (eess.AS); Computation and Language (cs.CL); Sound (cs.SD) |
| Cite as: | arXiv:2610.35581 [eess.AS] |
| (or arXiv:2610.35581v1 [eess.AS] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35581 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Guoqiang Hu [view email]
[v1]
Mon, 26 Oct 2026 15:14:03 UTC (243 KB)
来源：arXiv cs.CL · arxiv.org