arXiv cs.CL· Wei Chu, Yuanzhe Dong, Ke Tan, Dong Han, Yichao Zhou, Ruchao Fan, Bingshen Mu, Jingbei Li, Vishwas Shetty, Sarthak Bisht, Ziyue Qiu, Massa Baali, Rita Singh, Bhisha Raj·· 9 天前AI 评分24
FA-Bench：面向强制对齐与 ASR 的干净及噪声语音音素级与词级时间戳准确率基准
FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech
AI 导读
FA-Bench 是一个统一评测强制对齐与 ASR 时间戳准确率的开放框架，固定了文本归一化、数据划分、音素映射与评分脚本，在干净及四种退化语音上覆盖 21 个开源模型和 9 个商业 API。它采用基于容差的 F1 作为主指标，消除了标准 MAE 在对话语音中造成的 9% 至 14% 分数虚高，并发现 Whisper 时间戳平均偏早约 150 ms，多个商业 ASR API 偏晚超过 50 ms。
正文 · 原文
Authors:Wei Chu, Yuanzhe Dong, Ke Tan, Dong Han, Yichao Zhou, Ruchao Fan, Bingshen Mu, Jingbei Li, Vishwas Shetty, Sarthak Bisht, Ziyue Qiu, Massa Baali, Rita Singh, Bhisha Raj
Abstract:Forced alignment estimates the timestamps of each word, phone or character in speech given its transcript. Published comparisons normalize transcripts, split the data and match boundaries differently, so their numbers cannot be read together. We present FA-Bench, an open framework that fixes those choices once and releases the code, splits, phone mapping, text normalization and scoring script, with results published periodically. Track 1 gives every aligner the reference transcript and Track 2 gives it a recognizer's output, on the same audio, clean and degraded four ways, with 21 open models and 9 commercial APIs under a unified protocol. We score every boundary of an utterance and check the two labels beside it, so a word the recognizer missed or invented is charged. Using a tolerance-based F1 as our primary metric eliminates the 9% to 14% score inflation that standard MAE causes on recognition-dependent systems in conversational speech. We then group boundaries by position and how many adjacent words were recognized correctly, which shows where a system lost the score. We discovered systematic bias in how current systems time words, with Whisper about 150 ms early and several commercial ASR APIs over 50 ms late. Code and results are at this https URL
| Subjects: | Computation and Language (cs.CL); Performance (cs.PF) |
| Cite as: | arXiv:2609.32396 [cs.CL] |
| (or arXiv:2609.32396v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.32396 arXiv-issued DOI via DataCite |
Submission history
From: Wei Chu [view email]
[v1]
Sat, 26 Sep 2026 09:16:06 UTC (23 KB)
[v2]
Thu, 8 Oct 2026 17:57:56 UTC (23 KB)
来源：arXiv cs.CL · arxiv.org