arXiv cs.AI· Linhai Ma, Yan Wang, Lingfei Qian, Yupeng Cao, Victor Gutierrez Basulto·· 3 天前AI 评分32
工具调用参数校验中的推导鸿沟：LLM 评审器为何依赖参考文本
Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Arguments
AI 导读
研究发现，LLM 工具调用评审器在输入中缺少参考文本时，会降低对正确参数值的接受率，这一现象被称为“推导鸿沟”，在包括 CLM 和 Jev 在内的全部七款评审器上均出现。
正文 · 原文
Abstract:Tool-call judges evaluate argument values proposed by LLM agents. We find that they accept a correct value less often when their input holds no reference for it, that is, no text that says what the value should be. This happens when the value is a default value from the function schema that the user did not mention, when it has to be inferred, or when the text it comes from was cut from the judge input. We call this drop in acceptance the derivation gap. It appears across all seven judges we evaluate, including System One models CLM and Jev. The gap is not only a truncation artifact: correct default values have the largest gap on a benchmark with no truncation. A controlled source-relocation test shows that judges follow the reference in their input: they favor a visible incorrect value over a correct value whose source is hidden. We introduce DeRef (Derived Reference): a separate model reads the full history without seeing the candidate and writes a reference for the judge. Across three benchmarks, DeRef improves overall argument AUROC for all judges tested. On the controlled test, it raises AUROC from chance or below to between 98.40 and 99.59.
| Comments: | 28 pages, 6 figures, 18 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.18497 [cs.AI] |
| (or arXiv:2610.18497v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18497 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Linhai Ma [view email]
[v1]
Tue, 13 Oct 2026 14:52:59 UTC (153 KB)
来源：arXiv cs.AI · arxiv.org