arXiv cs.CL· Lukas Edman, Alexander Fraser·· 3 天前AI 评分22
Qwenfant：用简洁平行文本训练大字符级 BabyLM
Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text
AI 导读
面向 2026 BabyLM Challenge 多语言赛道，研究者基于 Qwen3 训练出 Qwenfant，提交时在该赛道排名第 1。方法用 LLM 摘要并翻译的数据提升数据质量，再叠加更大模型与字符级词表两项改动，二者均需 LoRA 式微调才能发挥全部潜力。
正文 · 原文
Abstract:This work concerns our submission for the Multilingual track of the 2026 BabyLM Challenge. We improve on our baseline using data that has been summarized and translated by an LLM, showing that higher-quality data can come from synthetic sources. We further improve with 2 simple changes: using a larger model, and using a character-level vocabulary. Both of these changes require LoRA-style fine-tuning for realizing their full potential. We dub our final Qwen3-based model Qwenfant, which, at the time of submission, ranks 1st in the Multilingual track.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19294 [cs.CL] |
| (or arXiv:2610.19294v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19294 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lukas Edman [view email]
[v1]
Tue, 13 Oct 2026 23:15:40 UTC (97 KB)
来源：arXiv cs.CL · arxiv.org