arXiv cs.CL· Siva Rajesh Kasa, Yasong Dai, Sumit Negi, Hongdong Li·· 11 小时前AI 评分34
SpecCTMC：扩散语言模型连续时间马尔可夫链的精确投机采样
SpecCTMC: Exact Speculative Simulation of Diffusion Language Model Continuous-Time Markov Chains
AI 导读
SpecCTMC 是一种无需训练、也无需独立草稿模型的投机采样器，用于加速扩散语言模型的精确连续时间采样。它复用已计算的模型预测提出连续 token 更新并并行评估中间状态，再经精确的接受-修正流程，证明可保持底层连续时间马尔可夫链在事件时间、生成顺序和 token 取值上的联合分布。
正文 · 原文
Abstract:Masked diffusion language models offer parallel text generation, but independently sampling multiple tokens can disrupt their dependencies and change the output distribution. Exact continuous-time sampling avoids this approximation, yet typically requires one sequential model evaluation per token. We introduce SpecCTMC, a training-free speculative sampler that reduces this sequential bottleneck without a separate draft model. SpecCTMC proposes successive token updates using previously computed model predictions, evaluates their intermediate states in parallel, and applies an exact acceptance-and-correction procedure. We prove that SpecCTMC preserves the joint distribution of event times, generation order, and token values of the underlying continuous-time Markov chain. It therefore introduces neither time-discretization error nor additional factorization error from parallel decoding. Experiments on LLaDA-8B-Instruct and Dream-v0-Instruct-7B across five benchmarks demonstrate 1.73-4.38x fewer sequential model evaluations than direct continuous-time sampling, while preserving the reference sampling distribution. Our code is available at: this https URL
| Comments: | 28 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.08362 [cs.CL] |
| (or arXiv:2611.08362v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08362 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yasong Dai [view email]
[v1]
Thu, 5 Nov 2026 17:41:20 UTC (120 KB)
来源：arXiv cs.CL · arxiv.org