亚秒级延迟下实时语音 AI 智能体的说话人嵌入模型基准测试

View PDF HTML (experimental)

Abstract:Real-time Voice AI agents must separate the active user from intelligible background speech that survives conventional noise suppression. Background sources include television audio, family members, and colleagues. Production denoisers remove non-stationary environmental noise effectively. Background speech shares the spectral characteristics of the target and passes through, causing false acceptances in downstream speaker verification (SV) and automatic speech recognition (ASR). We benchmark open-source speaker embedders TitaNet-Large, TitaNet-Small, ECAPA-TDNN, CAM++, and SpeakerNet under a strict sub-second latency budget. Evaluation combines a controlled benchmark with a weighted human-annotated production set. Our controlled benchmark is built from real calls in which a second real speaker is injected into silence intervals at modulated amplitude. We report three findings. First, denoising before embedding is generally harmful and raises equal error rate (EER) across architectures. Second, truncating audio below the full voice-activity segment degrades every model. Third, roughly half of the TitaNet embedding is redundant for this task. Trailing dimensions encode population attributes such as language and gender. Such attributes carry little discriminative value in a demographically homogeneous user base. We distill these results into deployment guidance. Verification should run on raw audio with full voice-activity chunks. Thresholds should favor precision over F1. Truncated embeddings reduce the cost of similarity search.

Comments:	13 pages, 6 figures, 4 tables. Accepted at AIST 2026
Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2611.09516 [cs.CL]
	(or arXiv:2611.09516v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2611.09516 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Abhishek Gautam [view email]
[v1] Fri, 6 Nov 2026 14:29:11 UTC (521 KB)