arXiv cs.AI· Lars Krupp, Daniel Gei{\ss}ler, Francisco M. Calatrava-Nicolas, Stefan Gerd Fritsch, Paul Lukowicz, Jakob Karolus·· 14 小时前AI 评分38
上下文真实成本：预测 LLM prefill 与 decode 阶段的计算时间和能耗
The Real Cost of Context: Predicting Compute Time and Energy Consumption in LLM Prefill and Decode Phases
AI 导读
研究系统测量了 LLM 在不同 prefill（输入）与 decode（输出）token 长度下的计算时间和能耗，发现 decode 阶段的计算时间与能耗随输出长度线性增长，而 prefill 阶段则随输入长度呈线性与二次方双重增长。
正文 · 原文
Abstract:The integration of Large Language Models (LLMs) into agentic architectures has substantially increased token consumption, raising concerns about their energy footprint. Yet, estimating the energy consumption of proprietary API-based models remains challenging, and existing approaches focus primarily on generated output tokens while overlooking the impact of the large input contexts characteristic of agentic workflows. To address this gap, we systematically profile compute time and energy consumption across a wide range of prefill (input) and decode (output) token lengths. Using generalized linear models, we show that decode compute time and energy scale linearly with output length, whereas prefill compute time and energy exhibit both linear and quadratic scaling with input length. These findings demonstrate that input context becomes a relevant factor for large-context workloads. Furthermore, we show that compute time serves as a reliable proxy for energy consumption when direct energy measurements are unavailable. Our mapping framework enables precise energy estimation and supports more sustainable deployment decisions for agentic AI.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39621 [cs.AI] |
| (or arXiv:2610.39621v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39621 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lars Krupp [view email]
[v1]
Thu, 29 Oct 2026 08:50:32 UTC (186 KB)
来源：arXiv cs.AI · arxiv.org