让 Diffusion LLM 多走几步：RePara 强化并行生成能力

A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs

AI 导读

针对扩散大语言模型在固定配额评估下数学与代码得分严重下滑（每步 16 token 时全部低于 15%）的问题，研究者提出 RePara，通过组相对策略优化强化短连续 token 块，并降低 rollout 与 log-probability 重算开销。

正文 · 原文

View PDF HTML (experimental)

Abstract:Parallel generation is widely regarded as an inherent advantage of diffusion large language models. Yet our fixed-quota evaluation of four representative models reveals severe degradation across mathematics and code, with all scores below 15% at 16 tokens per step. Token-level probes identify local predictive potential near the observed prefix, especially in mathematical text. We propose RePara, which reinforces short consecutive token blocks through group-relative policy optimization and reduces rollout and log-probability recomputation costs. On LLaDA and SDAR, \ours improves GSM8K and MATH-500 accuracy over JustGRPO under reduced denoising budgets, with a 1.96x training-step speedup in our LLaDA/MATH setup.

Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.18200 [cs.LG]
	(or arXiv:2610.18200v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.18200 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jiahao Kuang [view email]
[v1] Tue, 13 Oct 2026 11:37:04 UTC (424 KB)

来源：arXiv cs.AI · arxiv.org