arXiv cs.AI· Jannatun Noor, Athar Noor Mohammad Rafee, Md. Sadman Haque, Sreezon Das Gupta, Atiar Osman, Md. Abu Obaida Zishan·· 12 小时前AI 评分31
大语言模型云端部署综述：架构、编排与推理的系统性回顾
LLMs in Cloud: A Discussion on Architecture, Orchestration, and Inference
AI 导读
一项基于 PRISMA 2020 框架的系统性综述统一了硬件虚拟化、集群调度与运行时推理优化三个层面，覆盖 2024 至 2026 年的关键进展。
正文 · 原文
Abstract:The widespread deployment of foundational Large Language Models (LLM) across cloud datacenters marks a fundamental paradigm shift from stateless deep learning toward distributed, memory-intensive generative workloads. However, serving massive foundation models under stringent Service Level Objectives (SLO) exposes critical physical and system bottlenecks. These challenges include the persistent High Bandwidth Memory (HBM) wall, operational asymmetry between compute-bound prefilling and memory-bandwidth-bound decoding, explosive Key-Value (KV) cache expansion, prohibitive serverless cold starts, and hardware reliability vulnerabilities. Existing literature reviews remain layer-siloed, decoupling low-level accelerator virtualization from cluster orchestration and runtime inference, while largely overlooking defining advancements from 2024 to 2026. To address these limitations, this study presents a comprehensive, cross-layer systematic review guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 framework. We synthesize a holistic taxonomy unifying hardware virtualization, cluster-level scheduling, and runtime inference optimizations, including iteration-level continuous batching, prefill-decoding disaggregation, and hierarchical KV cache offloading. Furthermore, we evaluate resilient cloud-native service architectures, examining scale-to-zero serverless paradigms, fault-tolerant checkpointing against silent data corruption, and hardware-enforced Trusted Execution Environments (TEEs) for multi-tenant isolation. Finally, we formalize the multi-dimensional Pareto trade-offs governing throughput, latency, monetary cost, and confidentiality, outlining a forward-looking research roadmap spanning Compute Express Link (CXL) memory pooling, in-memory computing, and autonomous multi-agent control planes to guide future cloud LLM infrastructure.
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33475 [cs.DC] |
| (or arXiv:2610.33475v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33475 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Md Abu Obaida Zishan [view email]
[v1]
Sat, 24 Oct 2026 05:53:48 UTC (586 KB)
来源：arXiv cs.AI · arxiv.org