跳到正文
arXiv cs.CL· Anxo Perez, Jorge Gab\'in, Javier Parapar·· 10 小时前AI 评分31

小样本的 ROI:在 Token 预算下优化上下文学习

The ROI of Small Examples: Optimizing In-Context Learning Under Token Budgets

AI 导读

在固定 token 预算下,上下文学习应优先用更多短示例而非少量长示例。研究将候选示例按长度分桶,在 Beat the AI 和 Natural Questions 两个 QA 基准上用六个指令微调 LLM 实验,发现短示例通常最有效利用提示空间,该效应在 Beat the AI 上尤其明显,NQ 上增益较小且长上下文场景更早饱和。

正文 · 原文

View PDF HTML (experimental)

Abstract:In-context learning (ICL) improves large language model performance at inference time without updating model parameters. However, these gains are not free: in practical deployments, longer prompts directly increase inference cost and latency. Under a fixed token budget, prompt construction therefore becomes a resource-allocation problem. While prior work has often focused on how many examples to include in ICL, we study the complementary question of whether the available budget is better spent on a few long demonstrations or on a larger number of shorter ones. We study this aspect in question answering (QA), by organizing candidate demonstrations into length-based bins and constructing prompts from examples within the same length range under fixed budgets. Experiments on two QA benchmarks (\textit{Beat the AI} and \textit{Natural Questions}), using six instruction-tuned LLMs, show that shorter demonstrations are generally the most effective use of prompt space. This effect is especially strong on \textit{Beat the AI}, and remains visible on NQ, although gains are smaller and saturate earlier in the long-context setting. An additional natural language inference experiment on ANLI R3 finds that one of the two shorter bins performs best for five of six models, although longer demonstrations remain competitive. Overall, our results highlight demonstration granularity as an important factor in prompt design.
Comments: Accepted for EACL 27 Main Conference
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24971 [cs.CL]
  (or arXiv:2611.24971v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.24971

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Anxo Perez Anxo Perez [view email]
[v1] Mon, 16 Nov 2026 11:56:49 UTC (222 KB)

来源:arXiv cs.CL · arxiv.org