arXiv cs.CL· Mannon Ochilov, Oybek Narzullayev, Bahodir Zaripov·· 10 小时前AI 评分18
混合文字乌兹别克语文本的 OCR 后校正：字节级 ByT5 与经验同形异义噪声矩阵
Post-OCR Correction of Mixed-Script Uzbek Text: Byte-Level ByT5 with an Empirical Homoglyph Noise Matrix
AI 导读
针对乌兹别克语西里尔与拉丁字母混排文档中同形字符导致的 OCR 错误，研究提出结合字节级 ByT5 模型、基于 359 个人工分类错误事件估计采样权重的噪声生成器，以及 Levenshtein 约束的保守解码规则。
正文 · 原文
Abstract:In Uzbek official and technical documents, a large share of OCR errors is caused not by poor image quality but by characters that are visually identical across the two alphabets in use. Because the script transition is still under way, Cyrillic and Latin appear on the same page and often within a single sentence, and in our measurements the density of homoglyph errors on such pages is close to five times higher than in single-script text. We study this problem in a setting where the corpus of engineering-geological and ICT infrastructure reports may not leave the organisation, so every stage was executed on local hardware. The proposed system combines a byte-level ByT5 model with a noise generator whose sampling weights are estimated from 359 manually classified error events, together with a Levenshtein-bounded conservative decoding rule. On a Gold set of 5,204 line pairs it lowers the character error rate from 0.0578 to 0.0369, restores 91.3% of the homoglyphs and introduces only 84 new errors.
| Comments: | 7 pages, 2 figures, 7 tables. Accepted at the 10th International Conference on Future Networks and Distributed Systems (ICFNDS 2026), ACM ICPS. Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| ACM classes: | I.2.7; I.7.5 |
| Cite as: | arXiv:2611.13609 [cs.CL] |
| (or arXiv:2611.13609v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13609 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mannon Ochilov Dr. [view email]
[v1]
Tue, 10 Nov 2026 05:35:53 UTC (247 KB)
来源：arXiv cs.CL · arxiv.org