arXiv cs.AI· Shyam Agarwal, Olga Lvova, Agnia Sergeyuk, Anastasiia Serova, Timofey Bryksin, Thomas Zimmermann, Bogdan Vasilescu·· 10 小时前AI 评分49
JetBrains 研究：工程师 token 消耗量衡量的是什么？
The AI Era's Lines of Code? A Case Study on What an Engineer's Token Spend Measures
AI 导读
JetBrains 将 404 名工程师五周的编码智能体 token 用量与提交记录关联，发现 token 消耗主要因工程师而异，且高消耗者五周内基本稳定。在提交数相近的周里，消耗第 90 百分位是第 10 百分位的 39 至 240 倍；同一工程师消耗翻倍仅对应约 10% 的提交增长。现有工作指标只能解释工程师间消耗差异的 38%，因此 token 消耗更适合资源核算，而非衡量产出价值。
正文 · 原文
Abstract:Organizations use engineers' token spend to monitor AI adoption, set budgets, and measure productivity, but the metric's relationship with engineering output is unclear. We link five weeks of token usage in coding agents to the commits of 404 engineers at JetBrains, and analyze a company survey of 412 employees. We find that token spend varied mostly between engineers, and which engineers spent the most stayed largely the same over the five weeks. Among weeks with similar commit counts, the 90th percentile of spend was 39 to 240 times the 10th. Within an engineer, doubling spend was associated with about 10% more commits. Available work metrics explained 38% of the variation in spend between engineers and 13% of the week-to-week variation within an engineer. In the survey, respondents in technical roles reported in nearly nine of ten agent evaluations using agents for tasks beyond coding, such as analysis, research, and planning. Token spend therefore measures how much engineers use agents across many kinds of work and not the outcome and value of that work. We recommend using token spend at most for resource accounting, and evaluating agent-assisted work with evidence about tasks and outcomes.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13396 [cs.SE] |
| (or arXiv:2611.13396v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13396 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shyam Agarwal [view email]
[v1]
Tue, 10 Nov 2026 00:17:26 UTC (306 KB)
来源：arXiv cs.AI · arxiv.org