arXiv cs.AI· Zerui Yang, Miaomiao Dong, Quan Xue, Chaorui Zhang, Xuanyu Zheng, Xiyuan Wang, Haolin Ruan, Xueyan Niu, Guangcheng Song, Weixi Zhang, Wei Han, Bo Bai·· 4 小时前AI 评分47
上下文有代价:前缀缓存下 LLM 编程智能体的上下文增删研究
Context Has a Price: Adding and Removing Context for LLM Coding Agents under Prefix Caching
AI 导读
研究在 SWE-bench Verified、SWE-bench 和 Loc-Bench 上对开源模型进行超 2 万次运行,分别统计未缓存输入、缓存输入和输出 token 成本。
正文 · 原文
Authors:Zerui Yang, Miaomiao Dong, Quan Xue, Chaorui Zhang, Xuanyu Zheng, Xiyuan Wang, Haolin Ruan, Xueyan Niu, Guangcheng Song, Weixi Zhang, Wei Han, Bo Bai
Abstract:Coding agents repeatedly send their conversation history to a language model. Adding useful code may reduce search, while removing old tool outputs may shorten later prompts. But do these changes reduce the cost of solving a task? We study this as a measurement problem in more than 20,000 runs of open models on SWE-bench Verified, SWE-bench and Loc-Bench. We account separately for uncached input, cached input and output tokens. Three findings emerge. First, in our main evidence-package experiments, retrieved code reduces read and search commands by 18--34%, but increases cost by 33--42% when cached input costs one tenth of uncached input, without a measurable improvement in resolution. Second, removing old observations can either save money or increase cost: the same approach saves 19--36% without a cache discount but increases cost by 38--116% with a tenth-price cache. Third, shorter prompts do not guarantee cheaper runs because agents may respond by taking more steps. We also find that concurrent experiments can share cached prefixes and distort cost comparisons. These results show why context-management methods should be evaluated using both token prices and the agent's subsequent behavior.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22238 [cs.AI] |
| (or arXiv:2611.22238v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22238 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zerui Yang [view email]
[v1]
Tue, 17 Nov 2026 05:38:53 UTC (602 KB)
来源:arXiv cs.AI · arxiv.org