CEDR：用连续隐空间扩散做推理，GSM8K pass@1 达 68.97%

View PDF HTML (experimental)

Abstract:We introduce the Continuous Embedding Diffusion Reasoner (CEDR), a training and inference pipeline for continuous latent diffusion models that achieves strong performance on mathematical reasoning and code generation. We learn compact answer representations from three hidden layers of the autoregressive Qwen3-4B model. A staged curriculum first trains the ELF-based diffusion backbone with teacher prompt conditioning, then initializes a compact prompt encoder through imitation and jointly adapts both networks. This replaces the large teacher contextual prompt encoder at inference with a compact learned Transformer. The multilayer answer representation also enables asynchronous inference, with different components denoised under separate clocks. We further adapt DiffusionNFT to learned self-conditioning guidance, enabling reinforcement learning with verifiable rewards. Controlled ablations evaluate the effects of representation design, prompt training, and denoising schedules. CEDR outperforms recent continuous-diffusion baselines at comparable backbone scales, including before reinforcement learning. With a 638M-parameter denoising backbone and DMD2-style stochastic sampling, post-NFT CEDR-L achieves {pass@1, pass@8} accuracies of {68.97%, 80.82%} on GSM8K and {32.40%, 53.80%} on MATH500 at 64 denoising steps per sample, and {37.56%, 51.00%} on HumanEval+ and {26.96%, 39.92%} on MBPP+ at 128 steps per sample. Code and model weights are available at this https URL and this https URL, respectively.

Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2609.35694 [cs.AI]
	(or arXiv:2609.35694v3 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2609.35694 arXiv-issued DOI via DataCite

Submission history

From: Xiang Cheng [view email]
[v1] Mon, 28 Sep 2026 17:38:43 UTC (358 KB)
[v2] Tue, 29 Sep 2026 17:56:43 UTC (359 KB)
[v3] Tue, 20 Oct 2026 17:34:28 UTC (435 KB)