arXiv cs.AI· Mingyue Huo, Shivam Mehta, Bhavin Jawade, Yinghong Lan, Haoqi Li·· 12 小时前AI 评分31
SpeechCritic：仅用约 300 条人类偏好标注学习诊断式语音评判模型
SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences
AI 导读
SpeechCritic 提出一种参考条件化的跨语言诊断式语音评判方法，仅用约 300 条人类标注对比数据校准前沿音频语言模型，使其在判断两个候选语音优劣的同时指出音色、情感、时序等维度的差异及可听线索。
正文 · 原文
Abstract:Human speech conveys rich perceptual information, such as emotion and speaker identity, yet most automatic speech quality judges reduce it to a single naturalness score. We study diagnostic speech judges: given two candidates, a diagnostic judge decides which is better, along which perceptual dimensions (e.g., timbre, emotion, timing) they differ, and which audible cues support its decision. Learning such judges is challenging: expert annotation is costly, and simply prompting a frontier audio-language model to produce labels is unreliable: our probing reveals substantial errors and unstable instruction following. We introduce SpeechCritic, which learns a diagnostic judge in a reference-conditioned cross-lingual setting from only about 300 human-labeled comparisons. Rather than replacing the frontier model, SpeechCritic calibrates it with these labels: for each dimension, it selects the acoustic measurements that agree with human judgments, maps them to A/Tie/B probabilities, and passes these to the model as non-binding hints alongside the audio. Compared with the same model labeling without hints, this raises dimension-level agreement with humans by 6.3 points and cuts the mismatch with human Tie rates by 10.4 points. We then train a 7B judge on this supervision and find that different training signals shape different judge behaviors: SFT establishes the task, OPD transfers the teacher's dimension-level strengths and weaknesses, and RL helps most on clear-cut comparisons where human raters agree. Notably, human listeners also find that RL makes rationales cite more specific, localized acoustic cues, although it never directly rewards rationale text. Finally, we show that the pipeline is language-pair agnostic by instantiating it on both English-Japanese and English-Spanish. Together, these results demonstrate a path from limited human preferences to a diagnostic speech judge.
| Comments: | v2: update code url |
| Subjects: | Artificial Intelligence (cs.AI); Sound (cs.SD) |
| Cite as: | arXiv:2609.34582 [cs.AI] |
| (or arXiv:2609.34582v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.34582 arXiv-issued DOI via DataCite |
Submission history
From: Mingyue Huo [view email]
[v1]
Mon, 28 Sep 2026 08:24:45 UTC (1,694 KB)
[v2]
Fri, 16 Oct 2026 01:17:16 UTC (1,694 KB)
来源：arXiv cs.AI · arxiv.org