无投影进化策略实现连续提示词搜索

View PDF HTML (experimental)

Abstract:Continuous prompt search offers a computationally efficient alternative to conventional parameter tuning in natural language processing tasks. Nevertheless, its practical effectiveness can be significantly hindered by the black-box nature and the inherent high-dimensionality of the objective landscapes. Existing methods typically mitigate these challenges by restricting the search to a randomly projected low-dimensional subspace. However, the effectiveness and underlying motivation of the projection mechanism remain ambiguous. In this paper, we first empirically demonstrate that despite the prompt space possessing a low-dimensional structure, random projections fail to adequately capture this essential structure. Motivated by this finding, we propose a projection-free prompt search method based on evolutionary strategies. By directly optimizing in the full prompt space with an adaptation mechanism calibrated to the intrinsic dimension, our method achieves competitive search capabilities without additional computational overhead. Furthermore, to bridge the generalization gap in few-shot scenarios, we introduce a confidence-based regularization mechanism that systematically enhances the model's confidence in the target verbalizers. Experimental results on seven natural language understanding tasks from the GLUE benchmark demonstrate that our proposed approach significantly outperforms existing baselines.

Comments:	19 pages, including appendices
Subjects:	Computation and Language (cs.CL); Neural and Evolutionary Computing (cs.NE)
Cite as:	arXiv:2603.13786 [cs.CL]
	(or arXiv:2603.13786v2 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2603.13786 arXiv-issued DOI via DataCite

Submission history

From: Yu Cai [view email]
[v1] Sat, 14 Mar 2026 06:27:14 UTC (235 KB)
[v2] Thu, 5 Nov 2026 03:40:14 UTC (198 KB)