arXiv cs.CL· Zhiheng Qian, Junhao Du, Jing Peng, Hanqi Li, Kunyang Peng, Feng Xue, Kai Yu·· 11 小时前AI 评分20
ETC-Bench：面向评测侧转写文本规范化的多语言基准
ETC-Bench: A Multilingual Benchmark for Evaluation-Side Transcript Canonicalization
AI 导读
ETC-Bench 是首个面向评测侧转写文本规范化（ETC）的多语言基准，覆盖 12 种语言、181 种现象类型，每个实例为仅目标片段存在差异的最小对比对。研究同时提出 SENA 指标，结合独立规范化后的一致性精确匹配与 pair-relative BERTScore 约束来评估语义保持。基于该分类体系构建的 TACS 智能体在共享语言上优于五个基线配置。
正文 · 原文
Abstract:Written transcripts of the same speech can differ in ways that are not uniquely determined by the speech signal. We formulate evaluation-side transcript canonicalization (ETC), which maps multiple surface forms to a single canonical form when their differences are not determined by speech. To our knowledge, ETC-Bench is the first and most comprehensive multilingual benchmark for evaluation-side transcript canonicalization, covering 181 phenomenon types across 12 languages. Each instance is a minimal pair that shares a sentence context and differs only in a target span exhibiting one phenomenon. To evaluate both canonicalization success and semantic preservation, we propose Semantic-preserving Exact-match Normalization Accuracy (SENA), which combines exact agreement between independently canonicalized pair members with a pair-relative BERTScore constraint against information loss. We further introduce the Taxonomy-guided Agent Canonicalization Skill (TACS), which provides LLM agents with phenomenon descriptions and explicit canonicalization contracts. Experiments with five baseline configurations and three TACS-based agents show that the agents outperform the baseline systems on shared languages. ETC-Bench provides a systematic testbed for evaluation-side transcript canonicalization across languages.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.36527 [cs.CL] |
| (or arXiv:2610.36527v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36527 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhiheng Qian [view email]
[v1]
Tue, 27 Oct 2026 02:33:53 UTC (116 KB)
来源：arXiv cs.CL · arxiv.org