跳到正文
arXiv cs.AI· Zihao Wang, Ye Tian, Tajana Rosing, Nalini Venkatasubramanian·· 10 小时前AI 评分31

LEDGE:面向异构边缘设备分布式 LLM 服务的高效模拟器

LEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge Devices

AI 导读

LEDGE 是一款无需执行张量运算的轻量级模拟器,可快速细粒度评估异构边缘设备上的分布式 LLM 服务。在 10 台 Jetson 与 Raspberry Pi 设备、覆盖 4 个 LLM 家族 17 种模型-精度组合的测试平台上,Qwen 部署的请求级 MAPE 低于 3.6%,相比物理推理实现 135.3 倍聚合加速。

正文 · 原文

View PDF HTML (experimental)

Abstract:Evaluating distributed large language model (LLM) serving on heterogeneous edge devices requires exploring diverse model partitions, device placements, and network conditions, making repeated physical experiments costly. An efficient simulator for edge LLM serving must support diverse, fine-grained placement configurations, capture runtime behavior, and accurately predict execution times while keeping simulation overhead low. We present LEDGE, a lightweight simulator for fast, fine-grained evaluation of distributed LLM serving on heterogeneous edge devices without executing tensor operations. LEDGE constructs event graphs supporting non-uniform tensor and pipeline parallelism, estimates execution costs using reusable performance models calibrated from end-to-end measurements, and simulates shared-link contention. We evaluate LEDGE on a heterogeneous testbed of 10 Jetson and Raspberry Pi devices, covering 17 model--precision combinations across 4 LLM families. On Qwen deployments, LEDGE achieves request-level mean absolute percentage errors below 3.6% and a 135.3 times aggregate speedup over physical inference. Performance model calibrated on one LLM family yield 7.44% zero-shot end-to-end latency MAPE across three other families. Further simulations show how shared-link contention affects placement decisions and network topology constrains scaling. Across four edge environments, LEDGE completes 26,773 simulation runs in 83.8 minutes, enabling efficient exploration of deployment strategies under diverse resource conditions and serving objectives. The source code is publicly available at this https URL.
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24085 [cs.DC]
  (or arXiv:2611.24085v1 [cs.DC] for this version)
  https://doi.org/10.48550/arXiv.2611.24085

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zihao Wang [view email]
[v1] Wed, 18 Nov 2026 09:11:43 UTC (499 KB)

来源:arXiv cs.AI · arxiv.org