arXiv cs.CL· Roy Xie, Junlin Wang, Paul Rosu, Chunyuan Deng, Bolun Sun, Zihao Lin, Bhuwan Dhingra·· 6 天前AI 评分29
LLaMA/Qwen/Mistral 如何用潜在充分性信号实现动态上下文截断
Knowing When to Stop: Efficient Context Processing via Latent Sufficiency Signals
AI 导读
研究者提出"动态上下文截断"方法，让 LLM 在获取足够任务相关信息后自行终止处理。该方法通过轻量分类器检测特定注意力头编码的"充分性信号"，在六个 QA 数据集（最长 40K tokens）和 LLaMA/Qwen/Mistral 三个模型族（1B-70B）上实现平均 1.33x token 缩减，同时准确率提升 3.4%。研究还发现，小模型需探测充分性，大模型则可通过提示词展现内在自评估能力。
正文 · 原文
Abstract:Large language models (LLMs) process entire input contexts indiscriminately, which is inefficient when the information required to answer a query is localized within the context. We present dynamic context cutoff, a novel method enabling LLMs to self-terminate processing upon acquiring sufficient task-relevant information. Through analysis of model internals, we discover that specific attention heads inherently encode "sufficiency signals" -- detectable through lightweight classifiers -- that predict when critical information has been processed. This reveals a new efficiency paradigm: models' internal understanding naturally dictates processing needs rather than external compression heuristics. Comprehensive experiments across six QA datasets (up to 40K tokens) with three model families (LLaMA/Qwen/Mistral, 1B-70B) demonstrate 3.4% accuracy improvement while achieving 1.33x token reduction on average. Furthermore, our method demonstrates superior performance compared to other context efficiency methods at equivalent token reduction rates. Additionally, we observe an emergent scaling phenomenon: while smaller models require probing for sufficiency detection, larger models exhibit intrinsic self-assessment capabilities through prompting.
| Comments: | Accepted to NeurIPS 2025 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2502.01025 [cs.CL] |
| (or arXiv:2502.01025v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2502.01025 arXiv-issued DOI via DataCite |
Submission history
From: Roy Xie [view email]
[v1]
Mon, 3 Feb 2025 03:38:29 UTC (3,062 KB)
[v2]
Wed, 22 Oct 2025 21:46:56 UTC (2,270 KB)
[v3]
Sat, 7 Feb 2026 21:38:36 UTC (3,153 KB)
[v4]
Fri, 9 Oct 2026 10:31:04 UTC (3,157 KB)
来源：arXiv cs.CL · arxiv.org