arXiv cs.AI· Baiyu Su, Runlong Liao, Lizhang Chen, Qiang Liu·· 12 小时前AI 评分44
ReLF（Rectified Language Flow）：用整流流实现快速并行文本生成
Rectified Language Flow
AI 导读
研究者提出 Rectified Language Flow（ReLF），通过整流流将噪声传输到 token 嵌入来生成文本，结合 Hellinger-SVD 固定 token 嵌入、交叉熵训练的自条件引导与正则化 reflow。
正文
Abstract:Continuous-state flow and diffusion models offer a promising approach to fast, parallel language generation, but matching the generation quality of autoregressive models remains challenging. We present Rectified Language Flow (ReLF), a language model that generates text by transporting noise to token embeddings using rectified flow. ReLF combines several complementary design choices: fixed token embeddings constructed with Hellinger-SVD, self-conditioning guidance trained with cross-entropy, and regularized reflow for few-step generation. We further introduce singular logits, which combine an analytically computed term that drives commitment to a valid token embedding as noise vanishes with a bounded neural residual that learns contextual token preferences. With 64 function evaluations on OpenWebText, ReLF achieves lower generative perplexity than autoregressive baselines at matched diversity across all three model sizes tested, while also outperforming prior continuous flow methods in this setting. After two reflow rounds, ReLF still outperforms autoregressive baselines with 16 function evaluations while generating 1,024-token sequences 15-19 times faster at batch size 1.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.33811 [cs.AI] |
| (or arXiv:2610.33811v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33811 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lizhang Chen [view email]
[v1]
Sat, 24 Oct 2026 17:16:49 UTC (2,383 KB)
来源：arXiv cs.AI · arxiv.org