arXiv cs.CL· Yuxuan Chen, Yang Xiao, Jack Sun, Chak Sun, Brick Li, Sam·· 2 天前AI 评分26
全双工语音语言模型的参考引导轮流说话风格迁移
Reference-Guided Turn-Taking Style Transfer for Full-Duplex Spoken Language Models
AI 导读
研究者从全双工语音语言模型自身对话中提取出一个晚期层残差流方向——"主动性向量"，无需训练或标注即可读取并调控模型的说话决策，提前最多 800 ms 预判话轮起始。生成时缩放该向量可将语音占比从 6-12% 提升至 42-61%（不干预时为 21-30%），且跨对话、权重和语料保持一致。仅需一段简短参考录音即可将说话人的轮流说话风格迁移给模型，换用他人参考时 93-98% 的情况下匹配变差。
正文 · 原文
Abstract:Full-duplex spoken language models decide at every frame whether to take the floor, defining a conversational personality -- yet each set of weights encodes exactly one, which human speakers bracket rather than share. Existing controls mostly act from outside; none identifies the quantity that sets how much the model speaks. We identify it: a single late-layer residual-stream direction, the initiative vector, extracted from the model's own conversations without training or annotation. The same direction reads the decision, anticipating onsets up to 800 ms ahead, and shifts it: scaled during generation, it moves speech ratio from 6-12% to 42-61% (no steering: 21-30%), consistently across conversations, weights, and corpora. A short reference recording suffices to transfer a speaker's turn-taking style to the model, and the transfer is reference-specific: swapping in another speaker's reference worsens the match in 93-98% of cases, whereas the best uniform dose is indistinguishable from no steering. Turn-taking style is not frozen into the weights but a direction that can be read, shifted, and calibrated per person. Audio examples are available at this https URL.
| Comments: | 5 pages, 4 figures. Submitted to ICASSP 2027. Audio examples: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD) |
| Cite as: | arXiv:2610.20763 [cs.CL] |
| (or arXiv:2610.20763v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20763 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuxuan Chen [view email]
[v1]
Wed, 14 Oct 2026 18:12:58 UTC (522 KB)
来源：arXiv cs.CL · arxiv.org