arXiv cs.CY· Xudong Tang, Matthew Groh·· 11 小时前AI 评分42
VIPBench：语音克隆时代对齐人类感知的声纹识别基准
VIPBench: A Human-Aligned Benchmark for Voice Identity Perception in the Age of Voice Cloning
AI 导读
研究者提出 VIPBench，用 1,290 名听者对 100 位公众人物 9,800 对录音的 124,876 次“同人/不同人”判断，评估声纹模型与人类身份感知的对齐程度。
正文 · 原文
Abstract:Speaker-recognition benchmarks test how accurately a model recovers the speaker label of a recording, treating voice identity as a property of the person who produced it. However, these benchmarks do not evaluate how humans perceive voice identity. Whether a listener hears two voices as the same person has consequences for intellectual property rights, cybersecurity, and everyday life. We introduce the Voice Identity Perception benchmark (VIPBench), which contains 124,876 same-or-different judgments from 1,290 listeners on 9,800 recording pairs from 100 public figures. The pairs include real recordings, voice clones generated by a text-to-speech system, and morphs between two speakers' voices. We also define four tasks that evaluate the alignment between speaker-representation models and human identity perception. The first two tasks measure how well a model's similarity score predicts the proportion of listeners who respond "same" and the majority response. The other two measure whether a model and listeners find the same speaker pairs similar, and whether a model's relationship to listener judgments holds equally for real and synthetic recordings. We evaluate eleven publicly available speech models. Listener judgments diverge from speaker labels most often for voice clones: in 41% of responses, listeners judged a clone and a real recording of the same speaker to be different people. No model reaches the noise ceiling; the best model explains 61% of the explainable variance. Models trained with speaker labels align with listeners better than self-supervised models do, yet a model with a 12% error rate on speaker labels aligns with listeners as well as models with error rates below 2%. VIPBench enables the evaluation of speaker models against human perception on real, cloned, and morphed speech, and our results motivate training speaker models directly on listener judgments.
| Comments: | Accepted at NeurIPS 2026, Track on Evaluations and Datasets (poster). 30 pages, 12 figures, 22 tables. Dataset and code: this https URL |
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Multimedia (cs.MM) |
| Cite as: | arXiv:2611.05638 [cs.SD] |
| (or arXiv:2611.05638v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05638 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xudong Tang [view email]
[v1]
Tue, 3 Nov 2026 21:21:59 UTC (2,502 KB)
来源：arXiv cs.CY · arxiv.org