arXiv cs.CL· Justin Jung·· 10 天前AI 评分22
Noise Your Prompt：连续扩散语言模型训练中对条件提示词 token 加噪
Noise Your Prompt: Noising Conditioning Tokens in Continuous Diffusion Language Models
AI 导读
研究者对连续扩散语言模型训练中固定条件提示词 token 的常规做法做了单行改动：训练时也对条件提示词 token 加噪。
正文 · 原文
Abstract:We revisit a standard accepted practice in the continuous diffusion language model
literature of fixing conditioning prompt tokens clean during training.
We make a very simple modification: also noise the conditioning prompt tokens during training.
We demonstrate that under this modified training objective, we achieve better generalization
in combinatorial reasoning tasks such as Sudoku and N-Queens, with the largest gains on harder variants
($3.73\% \to 24.65\%$ solve rate on Sudoku Hard), and increased diversity of generated solutions ($50.60\% \to 73.79\%$ coverage on
10x10 N-Queens). We also show measurable improvements to natural language generation quality
in modest dataset regimes with Gigaword summarization, but notably demonstrate that gains do not
transfer to all natural language tasks (e.g open ended dialogue generation).
Our method is a single line change to the training objective, requires no additional inference costs by default,
and provides the flexibility of classifier-free guidance inspired guided sampling. Our
\href{this https URL} {code} is publicly available.
| Comments: | Published in Transactions on Machine Learning Research (TMLR), 2026 |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.09145 [cs.LG] |
| (or arXiv:2610.09145v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09145 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Transactions on Machine Learning Research (2026) |
Submission history
From: Justin Jung [view email]
[v1]
Tue, 6 Oct 2026 21:38:52 UTC (1,253 KB)
来源：arXiv cs.CL · arxiv.org