arXiv cs.AI· Juan Parra-Ullauri, Anthony Chiu, Hari Madhukumar, Zhaozhou Wu, Anderson Bravalheri, Alex Healing, Andy Corston-Petrie, Dimitra Simeonidou, Nektaria Efthymiou·· 12 小时前AI 评分31
InferSlice：面向电信广域网的 SLO 驱动 LLM 推理参考架构
InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area Network
AI 导读
BT 联合布里斯托大学与 NVIDIA 提出 InferSlice，一个在电信基础设施上实现 SLO 驱动 LLM 推理的参考架构，并在英国 JOINER 全国广域网完成验证。结果显示分层服务可形成不同的延迟-吞吐区间，权重流式加载能缩短扩容时间；复制对放置不敏感但受内存限制，流水线并行可容忍近距与中距跨站点跳转，而张量并行与预填充/解码分离在超出单一子网跳后失效。
正文 · 原文
Abstract:LLM inference is increasingly deployed as a core service for enterprise and interactive applications, but sovereignty, compliance, latency and resource-locality requirements can limit the suitability of centralised cloud deployments. Telco operators offer geographically distributed compute and network infrastructure for addressing these constraints. However, existing LLM serving frameworks are designed for tightly coupled data centres and provide limited support for coordinating geographically distributed and heterogeneous resources. We present InferSlice, a reference architecture for Service-Level Objective (SLO)-driven LLM inference over a telco infrastructure, integrating network slicing, GPU partitioning, elastic serving and observability. Led by BT in collaboration with the University of Bristol and NVIDIA, and realised on the JOINER UK national-scale wide-area network (WAN), InferSlice is evaluated on service differentiation across tiers, elastic scale-out, client-edge bandwidth as a bottleneck, and the reach of tensor, pipeline, data parallelism, and prefill/decode disaggregation over the WAN. Our results show that, on a general-purpose telco network, tiers yield distinct latency-throughput regimes, and weight streaming shortens scale-out time. Replication is placement-insensitive but memory-bound; pipeline parallelism tolerates near- and mid-cross-site hops, while tensor parallelism and disaggregation fail beyond a single subnet hop. We share practical insights from operating SLO-driven inference on a telco network, lessons learned and open questions, and hope to encourage further research on distributed and heterogeneous LLM inference systems.
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32638 [cs.DC] |
| (or arXiv:2610.32638v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32638 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Juan Marcelo Parra [view email]
[v1]
Fri, 23 Oct 2026 13:51:02 UTC (8,589 KB)
来源：arXiv cs.AI · arxiv.org