arXiv cs.CL· Luis A. Mercado-Campos, Alexis Palmer·· 2 天前AI 评分17
面向 Masewalkopa 的 Nawatl 单变体 NMT 专精化
Unfragmenting Nawatl NMT: Single-Variant Specialization for Masewalkopa
AI 导读
针对多变异体混合语料造成的“变体碎片化”问题，研究者基于 70,000 对多变异体数据，将 mT5-large 专精化到墨西哥 Santa Ana Tlacotenco 所说的 Nawatl 变体 Masewalkopa（nhn）。
正文 · 原文
Abstract:In minoritized (low-resource) language technologies, a common response to data sparsity is to aggregate text across multiple varieties and treat the combined data as a single corpus. However, mixed corpora of closely related indigenous variants can produce an artificial register that is accountable to no single speaker community, a structural failure we call variant fragmentation. This issue is acute for minoritized languages native to Mexico, such as Nawatl. Building on a 70,000-pair multi-variant foundation, we specialize an mT5-large checkpoint to Masewalkopa (nhn), the Nawatl variety spoken in Santa Ana Tlacotenco. We curate 2,665 parallel pairs, reserve all sentence-level data for evaluation, and incorporate 192 elder-transcribed target sentences via asymmetric backtranslation. A custom finite-state analyzer reduces unanalyzed surface forms from 82.0% to 19.9% and supplies functional syntax tags as plain-text prefixes, allowing the model to reuse existing subword embeddings rather than relying on frozen special tokens. The resulting LoRA-v9c-b model achieves 58.46 dev chrF++ and 58.20/57.90 directional test chrF++. Finally, CTranslate2 int8 quantization reduces CPU latency from 137.5 to 1.8-2.6 seconds per request, enabling cost-bounded Cloud Run deployment.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.22110 [cs.CL] |
| (or arXiv:2610.22110v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22110 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Luis Armando Mercado [view email]
[v1]
Thu, 15 Oct 2026 15:51:41 UTC (46 KB)
来源：arXiv cs.CL · arxiv.org