arXiv cs.CL· Hanyu Liao, Yijing Chen, Tao Ji·· 12 小时前AI 评分24
ARMOR-DP：面向多语言 BabyLM 的动态语言采样框架
When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM
AI 导读
ARMOR-DP 是一个模块化框架，用逐语言验证损失动态更新采样比例，诊断由表征、曝光、容量和目标压力造成的不平衡。实验显示等字符预算会给中文带来严重表征伪影，字节级编码可消除大部分，而静态中文保护随后可能让英语成为最弱语言。官方评测中课程动态 checkpoint 在更少中文 epoch 下优于匹配的静态中文保护基线，扩容则得到最强的内部 ARMOR-DP 系统。
正文
Abstract:Multilingual BabyLM systems usually fix language sampling proportions even though the weakest language can change during training. We present ARMOR-DP, a modular framework that diagnoses imbalance arising from representation, exposure, capacity, and objective pressure, and uses per-language validation loss to update sampling. Controlled experiments show that equal character budgets create a severe representation artifact for Chinese; byte-level encoding removes much of it, while static Chinese protection can subsequently make English the weakest language. The dynamic controller tracks these transitions and achieves a better score--balance tradeoff than fixed protection. On official evaluation, curriculum-dynamic checkpoints outperform a matched static Chinese-protective baseline while using fewer Chinese epochs; scaling capacity yields the strongest internal ARMOR-DP system. These results support a simple conclusion: multilingual weakness is a training-time state rather than a fixed property of a language. ARMOR-DP therefore separates diagnosis from intervention, allowing protection to follow the current bottleneck instead of a predetermined language label.
| Comments: | Accepted to the BabyLM Workshop at EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.33583 [cs.CL] |
| (or arXiv:2610.33583v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33583 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tao Ji [view email]
[v1]
Sat, 24 Oct 2026 10:23:25 UTC (5,716 KB)
来源：arXiv cs.CL · arxiv.org