arXiv cs.AI· Joon Ha Kim, Geon-Woo Kim, Anoop Rachakonda, Daehyeok Kim·· 13 小时前AI 评分31
Dooly：面向 LLM 推理模拟的配置无关、冗余感知性能分析工具
Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation
AI 导读
Dooly 通过一次推理遍历与污点传播标记输入维度来源，只对延迟数据库中缺失的算子做性能分析，实现配置无关、冗余感知的 profiling。在两种 GPU 平台、三种 attention 后端上，其 TTFT 模拟精度在 5% MAPE 内、TPOT 在 8% 内，12 个模型的 profiling GPU 时相比现有方法减少 56.4%。Dooly 已开源，可作为现有模拟器的即插即用后端。
正文 · 原文
Abstract:Selecting the optimal LLM inference configuration requires evaluation across hardware, serving engines, attention backends, and model architectures, since no single choice performs best across all workloads. Profile-based simulators are the standard tool, yet they hardcode their operation set to a specific configuration and re-profile every operation from scratch, making exploration prohibitively expensive. This cost stems from a missing structural understanding: every input dimension of each operation is fixed by the model configuration or determined by the incoming request. Many model-configuration values (e.g., head size, layer count) recur across models, so the same operation runs in many configurations; a single sweep over the request-dependent dimensions can serve them all. We present Dooly, which exploits this structure to achieve configuration-agnostic, redundancy-aware profiling. Dooly performs a single inference pass, labels each input dimension with its origin via taint propagation, and selectively profiles only operations absent from its latency database; stateful operations such as attention are isolated by reusing the serving engine's own initialization code, eliminating manual instrumentation. From this database, it builds latency regression models that serve as a drop-in backend for existing simulators. Across two GPU platforms, three attention backends, and diverse model architectures, Dooly achieves simulation accuracy within 5% MAPE for TTFT and 8% for TPOT while reducing profiling GPU-hours across 12 models by 56.4% compared to the existing profiling approach. We have open-sourced Dooly at this https URL.
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.07985 [cs.DC] |
| (or arXiv:2605.07985v3 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2605.07985 arXiv-issued DOI via DataCite |
Submission history
From: Joon Ha Kim [view email]
[v1]
Fri, 8 May 2026 16:44:47 UTC (766 KB)
[v2]
Thu, 21 May 2026 13:49:15 UTC (1,185 KB)
[v3]
Tue, 20 Oct 2026 18:41:32 UTC (774 KB)
来源：arXiv cs.AI · arxiv.org