arXiv cs.CL· Chung-En Sun, Linbo Liu, Ge Yan, Zimo Wang, Tsui-Wei Weng·· 12 小时前AI 评分31
LLM 智能体无需推理也能判断何时调用工具：When2Tool 基准与 Probe&Prefill 方法
LLM Agents Already Know When to Call Tools -- Even Without Reasoning
AI 导读
研究者提出 When2Tool 基准，覆盖 18 个环境（15 个单跳、3 个多跳），系统评测工具调用的必要性。探测发现工具必要性可从生成前隐状态线性解码，六个模型上 AUROC 达 0.89–0.96，远超模型自身语言化推理。基于此的 Probe&Prefill 用轻量线性探针加引导句预填充，在所有测试模型上减少 38% 工具调用，仅损失 1.7% 准确率。
正文 · 原文
Abstract:Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and latency, yet no existing benchmark systematically studies when a tool call is actually needed. We propose When2Tool, a benchmark of 18 environments (15 single-hop, 3 multi-hop) spanning three categories of tool necessity -- computational scale, knowledge boundaries, and execution reliability -- each with controlled difficulty levels that create a clear decision boundary between tool-necessary and tool-unnecessary tasks. We evaluate two families of training-free baselines: Prompt-only (varying the prompt to discourage unnecessary calls) and Reason-then-Act (requiring the model to reason about tool necessity before acting). Both provide limited control: Prompt-only suppresses necessary calls alongside unnecessary ones, and Reason-then-Act still incurs a disproportionate accuracy cost on hard tasks. To understand why these baselines fail, we probe the models' hidden states and find that tool necessity is linearly decodable from the pre-generation representation with AUROC 0.89--0.96 across six models, substantially exceeding the model's own verbalized reasoning. This reveals that models already know when tools are needed, but fail to act on this knowledge during generation. Building on this finding, we propose Probe&Prefill, which uses a lightweight linear probe to read the hidden-state signal and prefills the model's response with a steering sentence. Across all models tested, Probe&Prefill reduces tool calls by 38\% with only 1.7\% accuracy loss, while the best baseline at comparable accuracy only reduces 6% of tool calls, or achieves a similar tool call reduction but incurs a 5$\times$ higher accuracy loss. Our code is available at this https URL
| Comments: | Accepted to NeurIPS 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.09252 [cs.CL] |
| (or arXiv:2605.09252v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.09252 arXiv-issued DOI via DataCite |
Submission history
From: Chung En Sun [view email]
[v1]
Sun, 10 May 2026 01:37:40 UTC (686 KB)
[v2]
Wed, 20 May 2026 19:57:19 UTC (679 KB)
[v3]
Wed, 4 Nov 2026 03:01:20 UTC (681 KB)
来源：arXiv cs.CL · arxiv.org