arXiv cs.AI· Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim·· 1 天前AI 评分34
随机嵌入注入如何提升 LLM 推理多样性：RSP 无需训练即可媲美优化软提示
From Noise to Diversity: Random Embedding Injection in LLM Reasoning
AI 导读
研究者提出 Random Soft Prompts（RSP），完全跳过训练步骤，将按预训练嵌入表均值和方差拟合的各向同性高斯采样出的随机嵌入向量序列追加到 LLM 输入，在多个数学推理基准上达到与优化软提示相当的准确率。
正文 · 原文
Abstract:Recent soft prompt research has tried to improve reasoning by inserting trained vectors into LLM inputs, yet whether the gain comes from the learned content or from the act of injection itself has not been carefully separated. We study Random Soft Prompts (RSPs), which drop the training step entirely and append a freshly drawn sequence of random embedding vectors to the input. Each RSP vector is sampled from an isotropic Gaussian fitted to the entrywise mean and variance of the pretrained embedding table; the sequence carries no learned content, and yet reaches accuracy comparable to optimized soft prompts on math reasoning benchmarks in several settings. The mechanism unfolds in two stages: because attention has to absorb a never-seen-before random position, the distribution over the first few generated tokens flattens and reasoning trajectories branch, and as generation continues this influence dilutes naturally so the response commits to a single completion. We show that during inference RSPs lift early-stage token diversity and, combined with temperature sampling, widen Pass@N, the probability that at least one out of N attempts is correct. Beyond inference, we carry the same effect into DAPO training and demonstrate practical gains. Our contributions are: (i) RSP isolates the simplest form of soft prompt -- training-free, freshly resampled -- providing a unified lens for the structural effect of injection that variants otherwise differing in training and form all share; (ii) a theoretical and empirical validation of the underlying mechanism; and (iii) an extension from inference to training.
| Comments: | NeurIPS 2026 ICML 2026 Mechanistic Interpretability Workshop (Spotlight) |
| Subjects: | Artificial Intelligence (cs.AI) |
| MSC classes: | 68T05, 68T50 |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2605.11936 [cs.AI] |
| (or arXiv:2605.11936v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.11936 arXiv-issued DOI via DataCite |
Submission history
From: Heejun Kim [view email]
[v1]
Tue, 12 May 2026 10:47:20 UTC (959 KB)
[v2]
Thu, 23 Jul 2026 01:23:29 UTC (959 KB)
[v3]
Wed, 11 Nov 2026 04:07:13 UTC (975 KB)
来源：arXiv cs.AI · arxiv.org