arXiv cs.CL· T. Y. Tsui, Jiatao Gu, Lingjie Liu·· 9 天前AI 评分27
扩散模型与自回归模型的统一解码调度理论：过渡定律的格结构
The Lattice of Transition Laws
AI 导读
研究者将扩散模型、自回归（AR）模型及其中间形态统一描述为同一"损坏格"上的路径，并定义调度成本为并行步骤所丢弃的依赖关系。对于图上马尔可夫数据，零成本调度的最少步数等于图的 treedepth，该值随序列长度呈对数增长、随网格边长呈线性增长。在文本、图像和视频生成任务中，基于预训练权重估计的成对依赖核可在解码前预测不同调度的成本排名，且大部分预测得到验证。
正文 · 原文
Abstract:Diffusion and autoregression (AR) have long been seen as different categories of generative models, with diffusion specialising in continuous fields and AR specialising in discrete tokens. Recent work seeks to combine the advantages of the two models, and each hybrid fixes its decoding schedule by design. In this paper, we ask whether the performance of decoding schedules of one model can be predicted before decoding at a fixed number of steps. We describe diffusion, AR, and models in between as paths on one corruption lattice, and define the cost of a schedule as the dependence its parallel steps discard. The cost shows that the fewest steps of a zero-cost schedule are set by the geometry of the data, in the same way for tokens and for continuous fields. In particular, for data that are Markov on a graph and dependent along its paths, the fewest steps equal the graph's treedepth, which is logarithmic in the length of a sequence and linear in the side length of a grid. With fewer steps than the treedepth, every schedule pays a positive cost, whose ranking we predict before decoding with a kernel of pairwise dependence estimated from pretrained weights. Across text generation, image generation, and video generation, we verify most of the predictions about the rankings of different schedules under different metrics and benchmarks. This work therefore provides a design principle for decoding for future AR models, diffusion models, and anything in between. Our code is available at this https URL.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.11216 [cs.LG] |
| (or arXiv:2610.11216v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11216 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: T. Y. Tsui [view email]
[v1]
Thu, 8 Oct 2026 04:13:05 UTC (408 KB)
来源：arXiv cs.CL · arxiv.org