工具本身还是工具使用？追踪多模态智能体的能力增益

View PDF HTML (experimental)

Abstract:Tool-augmented multimodal agents show strong benchmark gains, often taken as evidence that agents have learned to use tools. We argue that this interpretation can be premature: a tool-call trace alone does not show whether the tool supplied answer-critical information. We study two representative ``thinking with images'' agents, Thyme and DeepEyesV2, across real-world understanding, OCR, chart understanding, and mathematical reasoning. Each agent is compared with its Tool-Free counterpart and with a Pure-Text Reasoner trained from the same source pool without tool-calling trajectories. Tool access yields little consistent aggregate improvement, does not reliably reduce generated-token cost, and leaves only a small tool-only solved set: 93% of DeepEyesV2's tool-solved problems and 96% of Thyme's are also solved by at least one non-tool setting. Mechanism ablations further show that the full tool-use loop does not consistently outperform either the tool-call format or the returned execution result alone. In the settings we study, the analyzed agents appear to learn tool-calling patterns more reliably than tool-contributed capabilities, suggesting that evaluation should distinguish tool availability from whether tools actually expand what agents can solve.

Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2606.02357 [cs.CV]
	(or arXiv:2606.02357v3 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2606.02357 arXiv-issued DOI via DataCite

Submission history

From: Jiawei Guo [view email]
[v1] Mon, 1 Jun 2026 15:04:25 UTC (1,002 KB)
[v2] Thu, 27 Aug 2026 10:57:33 UTC (1,002 KB)
[v3] Tue, 10 Nov 2026 05:14:31 UTC (1,002 KB)