研究：工具描述中的推销话术会显著影响 LLM 智能体的工具选择

View PDF HTML (experimental)

Abstract:AI agents often pick tools from registries, where each provider writes its tool's description; we ask whether sales language there changes which tool an agent picks. We built pairs of listings that differ in one way (praise, a verifiable specification, or list order) and asked two OpenAI models to call one tool. In a preregistered study, stacked praise (four kinds) raised a tool's pick rate by about 43 percentage points, matching or beating a verifiable specification. Praise also pulled some picks toward tools whose numeric limit was too small. With identical listings, the first-listed tool was picked about 72 points more often. Structured fields helped agents pick the capable tool; sales text beside them reduced or erased that gain. An exploratory study of five models shows the same direction; a "Sponsored" label dampened it, and a system-prompt warning moved one model. Registries could list limits as fields, hide sales text from agents, and randomize order. Stacked praise replicated on held-out domains; no single kind did after correcting for multiple tests. Preregistered results are provisional until blind phrase ratings are complete, and cover two small models.

Comments:	27 pages (9-page main text), 7 figures, 29 tables. Accepted at the NeurIPS 2026 Workshop "The Agentic Web: Networked, Continually-Adapting Agent Ecosystems"
Subjects:	Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); General Economics (econ.GN)
ACM classes:	I.2.7; I.2.11; K.4.4; J.4
Cite as:	arXiv:2605.23916 [cs.IR]
	(or arXiv:2605.23916v3 [cs.IR] for this version)
	https://doi.org/10.48550/arXiv.2605.23916 arXiv-issued DOI via DataCite

Submission history

From: Haochuan Wang [view email]
[v1] Sun, 12 Apr 2026 17:10:25 UTC (95 KB)
[v2] Wed, 30 Sep 2026 05:19:03 UTC (169 KB)
[v3] Fri, 16 Oct 2026 02:04:18 UTC (205 KB)