arXiv cs.CL· Eirik Botten Nicolaysen·· 11 小时前AI 评分18
挪威法规过时数值对比：进一步训练的 Borealis 2 与基座模型 Gemma 4 的预注册配对比较
Outdated numeric values from Norwegian statutes and regulations in a further-trained model and its base model: a preregistered paired comparison of Borealis 2 and Gemma 4
AI 导读
预注册研究对比 Borealis 2（挪威国家图书馆基于 Gemma 4 进一步训练）与 Gemma 4 在挪威法规过时数值上的表现，在 503 个主条目中两模型均可解释的 411 项上。
正文 · 原文
Abstract:We compare how often two released language models give outdated numeric values from Norwegian statutes and regulations: NbAiLab/borealis2-26b-a4b-preview (Borealis 2), which the National Library of Norway further trained from google/gemma-4-26B-A4B-it (Gemma 4), and Gemma 4 itself. The study was preregistered (doi:https://doi.org/10.5281/zenodo.23858965) and uses the closed norregel benchmark, scored deterministically against a value history. On the 411 of 503 primary items where both models gave an interpretable answer, Borealis 2 alone was outdated on 10 items and Gemma 4 alone on 11 (d = -0.2 percentage points, 95% cluster bootstrap interval [-2.7, +2.5], exact one-sided McNemar p = 0.668). By the preregistered rule this is no effect of interest: the whole interval lies within the smallest effect of interest of +/-10 percentage points. Among the secondary measures, only the share of answers giving a value that took effect after the question date differs after Holm correction; Gemma 4 gave such answers more often. An exploratory comparison of two smaller Borealis models had no discordant pairs and is not informative. The design compares released models and does not isolate the effect of any part of their training.
| Comments: | Preregistered: doi:https://doi.org/10.5281/zenodo.23858965. Data and code: doi:https://doi.org/10.5281/zenodo.23870450, this https URL. v2: title corrected; no change to data, analysis or results |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.02334 [cs.CL] |
| (or arXiv:2611.02334v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02334 arXiv-issued DOI via DataCite |
Submission history
From: Eirik Botten Nicolaysen Mr. [view email]
[v1]
Mon, 2 Nov 2026 00:31:48 UTC (11 KB)
[v2]
Tue, 3 Nov 2026 05:40:29 UTC (11 KB)
来源：arXiv cs.CL · arxiv.org