arXiv cs.AI· Haoyue Liu, Zhichao Wang, Huanyu Yan, Xiaoying Tang·· 9 天前AI 评分31
BudgetAPO：如何在紧预算下优化提示词？噪声自适应评估方案
How Should a Prompt Optimizer Spend a Tight Budget? BudgetAPO with Noise-Adaptive Evaluation
AI 导读
BudgetAPO 是一种面向紧预算场景的单阶段提示词优化器，通过短探测测量任务噪声来自适应调整评估切片大小，并用固定切片将每次接受/拒绝决策转为配对比较。在七个基准和五个被测模型上，BudgetAPO 在每个模型上均排名第一，并在 Holm 校正配对检验中击败所有基线；在 250 次调用预算下仅 13% 的运行返回原始种子提示词，而 GEPA 为 86%。
正文 · 原文
Abstract:Automatic prompt optimization (APO) has been widely employed to adapt large language models without updating their weights, yielding promising results. However, existing methods such as GEPA and OPRO assume hundreds to thousands of subject-model calls, far more than is practical behind paid, rate-limited APIs. Under tight budgets they fail in two ways: multi-stage pipelines can exhaust the budget and return the seed prompt unchanged, while single-stage methods compare candidates on fixed-size minibatches, regardless of each task's noise. As a remedy, we introduce BudgetAPO, a single-stage optimizer for the tight-budget regime. BudgetAPO incorporates (1) a noise-adaptive rule that sizes the evaluation slice to each task's noise, measured by a short probe; (2) a fixed slice that turns every accept/reject decision into a paired comparison; and (3) a reflective operator that rewrites reasoning strategy and output format jointly. Extensive results across seven benchmarks and five subject models demonstrate that BudgetAPO ranks first on every subject and beats every baseline under Holm-corrected paired tests, while returning the seed in 13% of runs at 250 calls against 86% for GEPA. On GPT-OSS-20B, GEPA needs 5 times as many calls to match \method's 100-call score.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.05671 [cs.AI] |
| (or arXiv:2610.05671v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.05671 arXiv-issued DOI via DataCite |
Submission history
From: Haoyue Liu [view email]
[v1]
Mon, 5 Oct 2026 01:34:14 UTC (6,677 KB)
[v2]
Thu, 8 Oct 2026 14:15:37 UTC (6,671 KB)
来源：arXiv cs.AI · arxiv.org