arXiv cs.AI· Paul J\"unger, Justin Lovelace, Linxi Zhao, Dongyoung Go, Kilian Q. Weinberger·· 14 小时前AI 评分31
SARDI：为扩散语言模型自增强检索的免训练 RAG 框架
Self-Augmenting Retrieval for Diffusion Language Models
AI 导读
SARDI 是一个面向扩散语言模型的免训练动态 RAG 框架，利用去噪过程中的推测性 token 预测提前引导检索。在五个多跳 QA 基准上，它以最高 4 倍吞吐量超越免训练的扩散与自回归检索基线。该方法与检索器无关，适用于任何具备推理能力的离散扩散语言模型，论文已被 ICML 2026 接收。
正文 · 原文
Abstract:Diffusion language models generate text by iteratively denoising an entire response in parallel, holding a noisy draft of the answer at every step. We show that these intermediate states surface salient entities early in the denoising trajectory, making them a useful lookahead signal for retrieval-augmented generation (RAG). We further demonstrate that grounding generation in retrieved evidence sharply reduces inter-token dependence, making RAG particularly well suited to parallel decoding. We exploit both through Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a dynamic RAG framework that uses speculative token predictions to guide retrieval during denoising. SARDI is training-free, retriever-agnostic, and applicable to any reasoning-capable discrete diffusion language model. Across five multi-hop QA benchmarks, SARDI outperforms training-free diffusion and autoregressive retrieval baselines at up to 4x higher throughput.
| Comments: | ICML 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.06474 [cs.CL] |
| (or arXiv:2606.06474v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.06474 arXiv-issued DOI via DataCite |
Submission history
From: Paul Jünger [view email]
[v1]
Thu, 4 Jun 2026 17:56:27 UTC (202 KB)
[v2]
Thu, 29 Oct 2026 16:45:03 UTC (262 KB)
来源：arXiv cs.AI · arxiv.org