arXiv cs.CL· Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina, Danial Namazifard, Annemette Brok Pirchert, Kenneth Enevoldsen, Lukas Galke Poech·· 13 小时前AI 评分42
Mimir：基于 HRM-Text 架构的 1.78B 分层推理模型，仅用合规数据训练即超越更大模型
Mimir: A Sub-2B Hierarchical Reasoning Model Trained on Permissible Data Punches Above Its Weight Class
AI 导读
Mimir 是一个 1.78B 参数的语言模型，基于 HRM-Text 循环深度 Transformer 架构，从随机初始化直接训练于 161 个数据集、每 epoch 70.5B token 的合规数据混合，续训版 Mimir v1.5 扩展至每 epoch 103B token。
正文 · 原文
Abstract:Training capable language models typically relies on web-scale pretraining followed by multi-stage post-training, creating substantial computational and data-governance barriers for settings in which training data must satisfy explicit permissibility constraints. We introduce Mimir, a 1.78B-parameter language model based on the Hierarchical Reasoning Model Text (HRM-Text) architecture, a recurrent-depth transformer whose layers are split into a low-level (L) and a high-level (H) stack that are applied repeatedly in an interleaved schedule. Mimir is trained from random initialization directly on a curated mixture of instruction, reasoning, tool-use, multilingual, and synthetic data: 161 datasets and 70.5B tokens per epoch for Mimir v1, extended to 103B tokens per epoch for its continued-training successor Mimir v1.5, with synthetic replacements for sources that do not satisfy permissibility criteria. Across 20 benchmarks in English, Danish (a low-resource language with ~6M speakers), and Math & Code, Mimir v1.5 attains the highest average score of all sub-2B models in every domain, outperforming both HRM-Text 1B and the looped transformers Ouro 1.4B and Ouro 2.6B, and Mimir matches or exceeds the performance of models of up to 9B parameters, such as Qwen 3.5 9B and Gemma 4 E4B, on the English and Danish averages. Two independent memorisation audits indicate low memorisation risk despite multi-epoch training. Inference-time ablations show that the number of L-stack applications can be changed with graceful degradation, whereas changing the number of H-stack applications breaks the model. Our results provide evidence that hierarchical recurrent language models can learn effectively from post-training-oriented, permissible data mixtures without conventional web-scale pretraining.
| Comments: | DFM Mimir, Hierarchical Reasoning Model, v2: added memorization audits |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.13517 [cs.CL] |
| (or arXiv:2608.13517v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.13517 arXiv-issued DOI via DataCite |
Submission history
From: Jacob Nielsen [view email]
[v1]
Thu, 13 Aug 2026 17:37:53 UTC (476 KB)
[v2]
Thu, 20 Aug 2026 18:12:30 UTC (253 KB)
[v3]
Thu, 22 Oct 2026 12:22:44 UTC (540 KB)
来源：arXiv cs.CL · arxiv.org