arXiv cs.CL· Haoyang Zheng, Yidong Ouyang, Blake Jianhang Chen, Tian Tan, Ziqian Lin, Liqian Peng, Alec Go, Guang Lin·· 10 小时前AI 评分17
扩散语言模型何时该重新审视其 token 承诺？PURL 提出可修正解码框架
When Should Diffusion Language Models Reconsider Their Commitments?
AI 导读
针对 Masked Diffusion Models（MDMs）并行采样易产生不一致预测的问题，研究者提出 PURL（Planned Unmasking-and-Revision Joint Learning）训练框架，将并行生成与选择性 token 修正统一起来。
正文 · 原文
This paper has been withdrawn by Haoyang Zheng
No PDF available, click to view other formats
Abstract:Masked Diffusion Models (MDMs) are well known for fast text generation via parallel decoding. However, independent parallel sampling can commit to inconsistent predictions, and less is known about how to selectively revise these early token commitments. In this work, we introduce $\underline{\mathrm{P}}$lanned $\underline{\mathrm{U}}$nmasking-and-$\underline{\mathrm{R}}$evision Joint $\underline{\mathrm{L}}$earning (PURL) for MDMs, a training-based framework that seamlessly unifies parallel generation with selective token revision. We derive a normalized joint transition whose likelihood factorizes into edit selection, unmask selection, and token prediction. These components are trained jointly on partially masked sequences constructed according to the model's own unmasking policy. During inference, all three decisions are supported by a single shared forward pass, enabling the model to decode masked tokens and revise noisy tokens. Across logic puzzles, mathematical reasoning, and code generation, PURL improves accuracy at matched unmask budgets, with relative gains up to 47.4% accuracy improvement and up to an $8\times$ reduction in scheduled decoding rounds. Beyond these gains, our analysis further reveals general principles of revisable decoding: token edits become more effective as context accumulates; early edits can steer unfinished trajectories, whereas later edits tend to remain localized; and effective edits therefore depend on both calibrating when to intervene and learning where and how to revise. Together, these suggest that effective parallel generation depends not on revising more tokens, but on making the right commitments revisable at the right time.
| Comments: | This paper is being withdrawn because agreement has not yet been reached regarding Google's authorization for its public release |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.11551 [cs.CL] |
| (or arXiv:2611.11551v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.11551 arXiv-issued DOI via DataCite |
Submission history
From: Haoyang Zheng [view email]
[v1]
Sun, 8 Nov 2026 18:49:48 UTC (1,083 KB)
[v2]
Tue, 10 Nov 2026 17:21:23 UTC (1 KB) (withdrawn)
来源：arXiv cs.CL · arxiv.org