arXiv cs.CL· Adrian Baule·· 11 小时前AI 评分31
掩码扩散即随机顺序吸附：一个可精确求解的并行解码模型
Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding
AI 导读
研究将掩码扩散模型的并行解码误差映射为驱动多粒子系统的统计物理问题，证明对任意数据，并行生成误差等于生成过程相对逆向掩码过程的熵产生。对马尔可夫数据，生成过程等价于带标签间隙的随机顺序吸附，并可对任意步数精确求解；多步极限下误差由 token 间互信息给出的热力学度量决定，最优调度是该度量的测地线。文本相关性越长，误差越集中在生成初期，先小步后加速的调度在路径误差和困惑度上均更优且所需步数更少。
正文 · 原文
Abstract:Masked diffusion models generate text by reversing a stochastic masking process. Unlike autoregressive LLMs, which generate text one token at a time, they can reveal many tokens in parallel, at the price of an error, since tokens revealed together are drawn independently. We show that this trade-off maps onto the statistical physics of driven many-particle systems. (i) For any data, the error of parallel generation is the entropy production of the generation process relative to the reversed masking process. (ii) For Markov data, generation is a random sequential adsorption process with labelled gaps, which we solve exactly for any number of steps. (iii) For many steps, the error becomes the dissipation of a slowly driven system: the unmasking schedule plays the role of the driving protocol, the number of steps that of its duration, and the error is set by a thermodynamic metric that we obtain in closed form from the mutual information between tokens. Optimal schedules are geodesics of this metric. The exact solution shows how correlations in the text shape both the error and the quality of the output. The longer the correlations, the more the error concentrates at the start of generation. Schedules that start with small steps then gain more, both in path error and in perplexity, and need fewer steps to reach their asymptotic behaviour.
| Comments: | 14 pages (incl. appendices), 3 figures |
| Subjects: | Statistical Mechanics (cond-mat.stat-mech); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.07845 [cond-mat.stat-mech] |
| (or arXiv:2611.07845v1 [cond-mat.stat-mech] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07845 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Adrian Baule [view email]
[v1]
Thu, 5 Nov 2026 11:23:40 UTC (50 KB)
来源：arXiv cs.CL · arxiv.org