arXiv cs.AI· Tommaso Salvatori, Matteo Forasassi, Alexis Toumi, Giovanni de Felice·· 3 天前AI 评分30
Tiny Recursive Message Passing：16k 参数模型在 Sudoku-Extreme 上达约 99% 测试准确率
Tiny Recursive Message Passing
AI 导读
研究者将递归 Transformer 的深度监督、截断递归与测试时离散演绎格等训练流程迁移到消息传递网络，发现两者互补，可在 Sudoku-Extreme 上以 16k 参数模型取得约 99% 测试准确率，H100 训练约 10 分钟、CPU 约 9 小时。
正文 · 原文
Abstract:Message passing neural networks have shown remarkable performance in algorithmic and constraint reasoning, due to their ability to encode the structure of the task directly in the architecture. Recent work has shown that better results can be achieved using recursive transformers trained with deep supervision and truncated recursions, which make extremely deep loops stable to train, and equipped with test-time procedures such as discrete deduction lattices. In this work, we apply the same procedures to message passing networks, and show that the two ingredients are complementary, and lead to parameter efficient models: we obtain about $99\%$ test accuracy on Sudoku-Extreme with a model of $16$k parameters that trains in about $10$ minutes on an H100, or in $9$ hours on a CPU. We further show that the cost of truncating gradients grows with the number of rounds information needs to cross the graph, making the choice of wiring a trade-off between robustness to truncation and transfer across problem sizes. We conclude by showing that our models transfer zero-shot to larger instances than those seen during training, outperforming popular neural baselines on N-queens and Boolean satisfiability with a fraction of their parameters.
| Comments: | 27 pages |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18649 [cs.LG] |
| (or arXiv:2610.18649v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18649 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tommaso Salvatori [view email]
[v1]
Tue, 13 Oct 2026 16:13:31 UTC (1,856 KB)
来源：arXiv cs.AI · arxiv.org