arXiv cs.AI· Bole Ma, Ayesha Afzal, Jan Eitzinger, Gerhard Wellein·· 14 小时前AI 评分38
LLM 解码阶段的功耗上限幻觉：跨注意力架构的分阶段能耗表征
The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures
AI 导读
在 NVIDIA H200 上用 vLLM eager 执行测试 GQA、MLA、Gated DeltaNet 和 Mamba2 四种注意力架构后发现，功耗上限对主导 LLM 服务的自回归解码几乎无效：解码内核等待 HBM、GPU 在 kernel 启动间隙空闲，功耗不足 700W 额定值的一半，各上限下吞吐差异仅几个百分点。
正文 · 原文
Abstract:Power capping is a common GPU energy lever in LLM serving, and it appears to work: the cap is set, the GPU reports drawing less than it, and the energy budget is met. We show the cap does little for the phase that dominates LLM serving: autoregressive decode. Across four attention paradigms (GQA, MLA, Gated DeltaNet, and Mamba2) on an NVIDIA H200 under vLLM eager execution, decode kernels wait on HBM and the GPU idles between kernel launches, so decode draws less than half of the GPU's 700W rating and throughput stays within a few percent under every cap we set. The cap engages only when a large batch runs a long generation. Locking the SM clock instead lowers power without slowing memory-bound decode and recovers up to 32% of decode energy at minimal throughput loss. We identify three architecture-dependent frequency-scaling classes and find that, once prefill is measured without prefix caching, the recurrent architectures pay no prefill penalty: their linear-time mixing undercuts quadratic attention on long prompts, so their efficient decode adds to the prefill saving and lowers total request energy from the first output token at large batch.
| Comments: | Accepted by PPAM 2026 (16th International Conference on Parallel Processing and Applied Mathematics) |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Performance (cs.PF) |
| Cite as: | arXiv:2605.11999 [cs.DC] |
| (or arXiv:2605.11999v2 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2605.11999 arXiv-issued DOI via DataCite |
Submission history
From: Bole Ma [view email]
[v1]
Tue, 12 May 2026 11:48:16 UTC (237 KB)
[v2]
Thu, 29 Oct 2026 15:43:01 UTC (158 KB)
来源：arXiv cs.AI · arxiv.org