arXiv cs.CL· Mingge Xu·· 2 天前AI 评分31
文本才是瓶颈：在 2B 参数以内小预算复现 LLM 打日本麻将
Text Is the Bottleneck: A Small-Scale Replication of LLM Game Playing in Riichi Mahjong
AI 导读
一项小预算复现研究发现，在同等 10 万决策训练集上，2.6M 参数的卷积棋盘编码器与强 AI Mortal 的决策一致率为 80.6%，高于所有受训文本提示 LLM（最佳 77.8%）。
正文 · 原文
Abstract:We replicate the riichi-mahjong track of "Can Large Language Models Master Complex Card Games?" (arXiv:2509.01328) under a drastically smaller budget: models of at most 2B parameters, one 16 GB consumer GPU, about five GPU-days. (1) With matched data and labels, text is the bottleneck: on the same 100k-decision training set, a 2.6M-parameter convolutional board encoder agrees with the strong AI Mortal on 80.6% of test decisions, above every text-prompt LLM we train (best 77.8%), and places better at the table (2.85 over 1,000 games vs. ~3.0 for the LLMs; paired difference at n=100 not significant). Scaled to 1.71M decisions, a 4.9M-parameter Transformer reaches 94.7% agreement, beats the 0.8B LLMs on identical seeds (-0.45 ranks, sign test p=0.005), and is statistically equivalent to parity against three Mortal champions (TOST within 0.10 ranks, p=6e-4); parity against a teacher-opponent is roughly the ceiling of pure imitation. (2) Training toward the teacher raises teacher agreement more than a 2.5x parameter increase does (+2.5 vs. +1.5 points); on human-record agreement both effects are +1.0, and neither is visible at the table. (3) Few-point agreement differences do not resolve into playing strength within a family: +2.5 agreement points coincide with a paired placement difference of +0.02 +/- 0.12 ranks (p=1.0), and +2.0 points at 2B with -0.06 (p=0.37) - both below the ~0.25-rank resolution of 100 paired games. We also document a paper-code mismatch in the original winner filter, reconstruct its two private engine patches, and show that a divide-and-conquer pipeline makes template-generated Japanese rationales 93% sentence-exact (43% without facts). Per Tenhou's terms we release no trained weights - only code, configurations, and evaluation protocols.
| Comments: | 12 pages, 3 figures, 2 tables. Replication of the riichi-mahjong track of arXiv:2509.01328. Code, configurations and ~3,200 arena game logs released; no trained weights (Tenhou data terms) |
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.34791 [cs.CL] |
| (or arXiv:2610.34791v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34791 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mingge Xu [view email]
[v1]
Mon, 26 Oct 2026 03:32:44 UTC (74 KB)
来源：arXiv cs.CL · arxiv.org