arXiv cs.CL· Hailay Kidu Teklehaymanot, Debela Desalegn Yadeta, Wolfgang Nejdl·· 12 小时前AI 评分22
VEXMLM：扩展 XLM-R 词表以改善阿姆哈拉语与提格里尼亚语的 Ge'ez 文字分词
Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya
AI 导读
VEXMLM 是面向阿姆哈拉语和提格里尼亚语的 XLM-R 词表扩展版本，新增 30k 个 Ge'ez 文字子词，并将新嵌入初始化为预训练嵌入的均值。其分词器 fertility 相对 XLM-R 分别降低 28.0%（阿姆哈拉语）和 45.9%（提格里尼亚语），下游命名实体识别略有提升，但抽取式问答低于 XLM-R，情感分析持平。
正文 · 原文
Abstract:Multilingual pre-trained language models such as XLM-R perform well for major languages but struggle with low-resource Ge'ez-script languages, largely because Latin-script-centric tokenizers split their words into many subwords. We introduce VEXMLM, a vocabulary-extended variant of XLM-R targeting Amharic and Tigrinya. We train language-specific SentencePiece tokenizers on monolingual corpora, extend XLM-R's vocabulary with 30k Ge'ez-script subwords, and initialize each new embedding to the mean of the pretrained embeddings. VEXMLM undergoes two-stage training: (1) continued masked language modeling on the monolingual corpora and (2) supervised fine-tuning on question answering and named entity recognition (Amharic and Tigrinya) and sentiment analysis (Amharic). VEXMLM lowers tokenizer fertility below that of XLM-R and Glot500 on both languages, by 28.0% (Amharic) and 45.9% (Tigrinya) relative to XLM-R. Downstream, it modestly improves named entity recognition over XLM-R, scores below XLM-R on extractive question answering, and is comparable on sentiment analysis. An ablation on Tigrinya NER shows that vocabulary expansion alone lowers accuracy on out-of-vocabulary words (words that XLM-R's tokenizer cannot represent or splits into more pieces than the expanded tokenizer), and that continued pretraining is required for the expanded model to exceed the baseline. Vocabulary expansion thus makes Ge'ez-script tokenization substantially more efficient, while its downstream benefit depends on the task and on adapting the new embeddings through continued pretraining. Resources: GitHub repository | Hugging Face model.
| Comments: | v4 replaces all results of v1 and v2, which could not be traced to experimental outputs. The results reported here come from runs carried out in August 2026, and its conclusions differ from those of v1 and v2 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.15209 [cs.CL] |
| (or arXiv:2607.15209v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.15209 arXiv-issued DOI via DataCite |
Submission history
From: Hailay Kidu Teklehaymanot [view email]
[v1]
Thu, 16 Jul 2026 17:08:28 UTC (252 KB)
[v2]
Tue, 18 Aug 2026 08:58:42 UTC (243 KB)
[v3]
Mon, 28 Sep 2026 16:27:34 UTC (278 KB)
[v4]
Fri, 23 Oct 2026 12:55:10 UTC (278 KB)
来源：arXiv cs.CL · arxiv.org