arXiv cs.AI· Xinsong Feng, Chenan Wang, Leshu Tang, Johnny Liu, Dong Li, Da Li, Haipeng Chen·· 12 小时前AI 评分29
DELA：为扩散语言模型学习可控编辑
DELA: Learning Controllable Edits for Diffusion Language Models
AI 导读
研究者提出 Diffusion Editing Language Adapter（DELA），一个在冻结扩散语言模型上实现可控草稿精修的插件框架，通过 Insert、Delete、Remask、Stop 四种显式编辑动作决定何时何地修改草稿。
正文 · 原文
Abstract:Diffusion language models (DLMs) support bidirectional denoising and iterative refinement, but generation remains largely controlled by token-level masking schedules. This leaves draft-level refinement implicit and limits explicit control over when to insert missing content, delete redundant spans, revisit uncertain regions, or stop editing. We introduce Diffusion Editing Language Adapter (DELA), a plug-in framework that enables controllable draft refinement with a frozen DLM. DELA operates on clean, variable-length drafts using four explicit editing actions: Insert, Delete, Remask, and Stop. The controller determines when and where to edit the draft, while the frozen DLM fills masked regions through denoising, separating high-level edit control from token-level generation. A naive single-phase editor would be computationally expensive because arbitrary interior edits can disrupt prefix stability and limit reliable KV-cache reuse. To address this, DELA uses a two-phase decoding procedure: a fast, append-only drafting phase that supports KV-cache reuse, followed by a global refinement phase that enables targeted non-monotonic edits to the full draft. We train the controller using a staged curriculum that begins with learning local edit priors from synthetic recovery tasks and mined actions that improve draft quality. Subsequent stages refine edit selection through rollout-based outcome preferences and alignment with downstream task feedback. Experiments on mathematical reasoning and code generation show that DELA achieves gains of up to 5.5 percentage points over direct diffusion decoding while keeping the backbone frozen. DELA thus provides a modular approach to explicit draft-level control in diffusion language models.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32096 [cs.AI] |
| (or arXiv:2610.32096v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32096 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xinsong Feng [view email]
[v1]
Fri, 23 Oct 2026 02:57:27 UTC (228 KB)
来源：arXiv cs.AI · arxiv.org