arXiv cs.AI· Hung-Chun Lin, Ting-Wei Hsu, Chung-En Ho, Ahmed Saeed·· 1 天前AI 评分27
ARK：为分离式 LLM 推理中的 KV Cache 传输规避路由冲突
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
AI 导读
ARK 是一种分布式大象流路径预留机制，通过协调发送方选择源端口，使并发流的哈希映射到不同 spine 链路，从而避免 ECMP 哈希冲突，且无需改动交换机或接收端重排数据包。在中等负载的包级 RDMA 模拟中，ARK 将平均和 P95 FCT 分别最多降低 27.3% 和 34.0%，平均 TTFT 最多降低 12.9%。该成果发表于 ACM SIGCOMM 2026。
正文 · 原文
Abstract:Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
| Comments: | 7 pages. Published in the Proceedings of the ACM SIGCOMM 2026 Conference (SIGCOMM '26) |
| Subjects: | Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) |
| Cite as: | arXiv:2610.41160 [cs.NI] |
| (or arXiv:2610.41160v1 [cs.NI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41160 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | In Proceedings of the ACM SIGCOMM 2026 Conference (SIGCOMM '26), pp. 2067-2073, 2026 |
| Related DOI: | https://doi.org/10.1145/3789240.3828750
DOI(s) linking to related resources |
Submission history
From: Chung-En Ho [view email]
[v1]
Fri, 30 Oct 2026 02:35:39 UTC (234 KB)
来源：arXiv cs.AI · arxiv.org