arXiv cs.CL· Nguyen Nguyen, Yongyi Zang·· 3 天前AI 评分27
semCER：面向 OCR 的严重度加权字符错误率
Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR
AI 导读
研究提出 semCER，在保留 CER 对齐方式与分母的基础上，由 LLM 评判为每个字素编辑赋予严重度权重，且无需分词。在英语、越南语和中文中，其评分逐条编辑地贴合读者评分，并能将读者认为最严重的编辑排在排序列表前列；在 CER 与 semCER 分歧的样本对上，英语和中文读者更常支持 semCER。研究还发布了 6100 万条已评判编辑、聚合读者评分及蒸馏评分器。
正文 · 原文
Abstract:Character Error Rate (CER), the standard metric for OCR and vision-language models (VLMs), charges a typo as much as a wrong name. We introduce semCER, which keeps CER's alignment and denominator and charges each grapheme edit a severity from an LLM judge, with no word segmentation. Normalized CER keeps the score of a whole transcription, where it tracks readers as well as semCER or better, and semCER breaks that score down into named edits. The charges track readers' ratings edit by edit in English, Vietnamese, and Chinese and put the edits readers rate most severe near the top of a sorted list. On pairs drawn where CER and semCER disagreed, readers side with semCER more often than with normalized CER in English and Chinese, and equally often in Vietnamese. Across 16 languages, the judge charges a wrong diacritic or tone mark almost nothing in Spanish but nearly a wrong letter's severity in Telugu. Where we can count real words, a general VLM's wrong letter forms one far above chance while a classical pipeline's stays near it. We release the 61 million judged edits, the aggregated reader ratings, and the distilled scorer at this https URL.
| Comments: | 26 pages, 7 figures, 14 tables. Under review at ACL Rolling Review |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| ACM classes: | I.2.7; I.7.5 |
| Cite as: | arXiv:2610.20461 [cs.CL] |
| (or arXiv:2610.20461v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20461 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Nguyen Nguyen [view email]
[v1]
Wed, 14 Oct 2026 16:44:22 UTC (610 KB)
来源：arXiv cs.CL · arxiv.org