arXiv cs.CL· Huimin Zeng, Yuxuan Lu, Jesse Cai, Eric Hanchen Jiang, Di Wu, Po-Nien Kung, Wenbo Hu, Ziyi Wang, Dakuo Wang, Jiaxin Pei, Kai-Wei Chang·· 11 小时前AI 评分44
AI 智能体如何浪费你的钱？统一框架量化 token 浪费
How Do AI Agents Waste Your Money?
AI 导读
研究者提出一个统一框架，用于跨异构智能体系统测量 token 浪费：记录并归一化完整轨迹，用 LLM-as-a-Judge 识别可删除片段，再映射回原轨迹做 token 级测量。
正文 · 原文
Authors:Huimin Zeng, Yuxuan Lu, Jesse Cai, Eric Hanchen Jiang, Di Wu, Po-Nien Kung, Wenbo Hu, Ziyi Wang, Dakuo Wang, Jiaxin Pei, Kai-Wei Chang
Abstract:AI agents may consume massive tokens across a wide range of tasks, but how many of those tokens are really necessary? Although prior work has shown that agents can incur unexpectedly high token costs, we still lack a systematic way to identify and measure token waste. In this work, we introduce a unified framework for measuring token waste across heterogeneous agent systems. Our framework records and normalizes complete agent trajectories, identifies removable spans using an LLM-as-a-Judge (LLMaJ), and maps the identified text spans back to the original trajectory for token-level measurement. We validate the identified waste through direct interventions by removing marked spans, resuming agent execution, and measuring changes in task outcomes and subsequent actions. In online pruning experiments, we remove 41-46% of the model's reasoning while largely preserving task performance across two distinct agent benchmarks. Using this framework, we analyze agents across multiple settings. Our analysis shows that token waste has a system-level structure that aggregate token usage alone cannot reveal. First, waste varies more across tasks and agent settings than across models. Second, reasoning and visible execution can exhibit different waste patterns. Third, different harnesses can introduce distinct sources of inefficiency even under the same model and task. Overall, our framework transforms token waste into an actionable optimization signal, enabling researchers and practitioners to train more efficient models, design better reasoning strategies, and optimize agent harnesses toward reducing unnecessary computation rather than merely reducing token usage.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.07238 [cs.CL] |
| (or arXiv:2611.07238v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07238 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Huimin Zeng [view email]
[v1]
Wed, 4 Nov 2026 23:43:03 UTC (1,300 KB)
来源：arXiv cs.CL · arxiv.org