arXiv cs.AI· Alessandro Breccia, Federica Gerace, Marco Lippi, Gabriele Sicuro, Pierluigi Contucci·· 12 小时前AI 评分31
研究：GPT-2 架构能否学会自然数素数分解生成的有根树算术序列
Testing Transformer Learnability on the Arithmetic Sequence of Rooted Trees
AI 导读
研究者用 GPT-2 架构从零训练，学习由自然数迭代素数分解映射成的有根平面树序列，词准确率约 0.4，显著高于隐马尔可夫模型基线。模型在远离训练区间的 10^13–10^15 测试块上性能保持稳定，并能将算术文本与保留词频但打乱顺序的合成序列区分开，且区分度随上下文扩大。模型还能识别算术上禁止的连续三个以上无平方因子整数序列。
正文 · 原文
Abstract:We study whether a transformer network can learn the deterministic sequence of trees generated by the iterated prime factorization of the natural numbers. Each integer is mapped into a rooted planar tree and the resulting sequence $\mathbb{N}\mathscr{T}$ defines an arithmetic text with measurable statistical structure. A transformer network (the GPT-2 architecture) is trained from scratch on the first $10^{11}$ elements and evaluated on next-token prediction and masked-token prediction tasks, with a Hidden Markov Model as baseline and a scaling analysis over context window, dataset size, vocabulary size and model size. The model reaches a word accuracy of about $0.4$, well above the baseline, and its performance remains stable on test blocks located at $10^{13}$-$10^{15}$, far beyond the training interval. Moreover, the likelihood assigned by the trained model separates the arithmetic text from two controls: synthetic token sequences that preserve the marginal token frequencies but not their order, with a separation that widens with the context, and sequences containing more than three consecutive square-free integers, a configuration that arithmetic forbids. These results indicate that the transformer captures regularities of the arithmetic text that go beyond its frequency profile.
| Comments: | 42 pages, 18 figures |
| Subjects: | Artificial Intelligence (cs.AI); Disordered Systems and Neural Networks (cond-mat.dis-nn); Mathematical Physics (math-ph); Number Theory (math.NT) |
| Cite as: | arXiv:2512.01870 [cs.AI] |
| (or arXiv:2512.01870v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2512.01870 arXiv-issued DOI via DataCite |
Submission history
From: Gabriele Sicuro [view email]
[v1]
Mon, 1 Dec 2025 16:51:38 UTC (423 KB)
[v2]
Sun, 2 Aug 2026 14:50:25 UTC (847 KB)
[v3]
Fri, 23 Oct 2026 13:41:47 UTC (1,346 KB)
来源：arXiv cs.AI · arxiv.org