arXiv cs.AI· Chengwei Zhou, Zhaoyan Jia, Massoud Pedram, Gourav Datta·· 14 小时前AI 评分31
Cord：面向掩码扩散语言模型的置信度锚定顺序残差解码
Cord: Confidence-Anchored Order Residual Decoding for Masked Diffusion Language Models
AI 导读
针对掩码扩散语言模型（dLLM）解码顺序问题，研究者提出 Cord——一种以置信度解码为稳定参照、通过几何残差学习何时偏离的轻量提示词条件策略。在 3 个 dLLM 和 5 个推理、代码与约束满足基准上，Cord 将平均任务成功率从置信度解码的 32.0% 提升至 48.2%，并优于各领域最佳固定残差。该方法仅增加 0.53M 可训练参数，无需额外主干前向，推理延迟开销为 5.39%。
正文 · 原文
Abstract:The decoding order of masked diffusion language models (dLLMs) can substantially affect generation quality, yet common decoders typically apply the same reveal rule throughout generation. Confidence provides a strong default, but it measures how decisively a position can be predicted now, not how useful revealing that position will be for subsequent decoding. We propose Cord, a lightweight prompt-conditioned policy that retains confidence decoding as a stable reference and learns when and how to depart from it through geometric residuals over generation direction, proximity to revealed context, and two-sided infilling. The residuals adapt to both the prompt and the evolving partial response. Rather than sampling a new policy action at every decoding step, Cord samples a single low-dimensional coefficient matrix per rollout, which induces an entire state-dependent reveal trajectory. This yields a contextual-bandit formulation trained directly from sequence-level rewards, without step-level credit assignment or backbone updates. Across three dLLMs and five reasoning, code, and constraint-satisfaction benchmarks, Cord improves average task success from 32.0% under confidence decoding to 48.2%, and further improves over the best per-domain fixed residual. Behavioral audits show that the learned residuals induce substantial, task-dependent changes in reveal locality and two-sided infilling. Cord adds only $0.53$M trainable parameters, requires no additional backbone forward passes, and incurs only 5.39% inference latency overhead.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39372 [cs.CL] |
| (or arXiv:2610.39372v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39372 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chengwei Zhou [view email]
[v1]
Thu, 29 Oct 2026 05:03:29 UTC (777 KB)
来源：arXiv cs.AI · arxiv.org