arXiv cs.CL· Zefang Liu, Chenyang Zhu, Sangwoo Cho, Xujun Peng, Rongqing Huang, Shi-Xiong Zhang, Sambit Sahu·· 12 小时前AI 评分27
RoleHear：面向半监督角色属性流式语音识别的伪标签方法
RoleHear: Pseudo-Labeling for Semi-Supervised Role-Attributed Streaming Speech Recognition
AI 导读
RoleHear 提出一种半监督流程，让单个 cache-aware 流式 transducer 通过词表内的角色 token 直接内联输出说话人角色，先输出快速轮次边界标记、几个词后再输出对应角色 token。
正文
Abstract:Streaming automatic speech recognition transcribes at low latency but produces an unattributed word stream, while real-time applications such as customer-service analytics need each word tied to a speaker role. Role attribution is usually a separate stage, either an offline or streaming speaker-diarization model aligned to the words or a text-based large language model applied to the transcript, and role labels for training are scarce. We present RoleHear, a semi-supervised pipeline that makes a single cache-aware streaming transducer emit roles inline through lightweight in-vocabulary role tokens: a fast turn-boundary marker followed by a per-role token a few words later. An offline transducer teacher, fine-tuned into a role-attributed recognizer, relabels unlabeled audio, and a multimodal large language model supplies silver role labels where gold annotations are unavailable. The streaming student is fine-tuned on the mixture in a single pass. Across customer-service calls with gold roles and earnings calls with silver roles, RoleHear attributes roles accurately while preserving transcription quality and narrowing the gap to the offline teacher.
| Subjects: | Computation and Language (cs.CL); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2610.33765 [cs.CL] |
| (or arXiv:2610.33765v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33765 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zefang Liu [view email]
[v1]
Sat, 24 Oct 2026 16:17:15 UTC (21 KB)
来源：arXiv cs.CL · arxiv.org