arXiv cs.CL· Shuhuai Li, Jianghao Lin, Dongdong Ge, Yinyu Ye·· 12 小时前AI 评分31
MoE-SpAc：异构边缘场景下基于推测激活效用的高效 MoE 推理
MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
AI 导读
MoE-SpAc 是一个面向异构边缘场景的 MoE 推理框架，将推测解码（SD）用作内存管理的前瞻传感器，并集成推测效用估计器、异构负载均衡器和异步执行引擎。在七个基准上，其 TPS 较 SOTA 的 SD 基线提升 42%，相对所有标准基线平均加速 4.04 倍。代码已开源。
正文 · 原文
Abstract:Mixture-of-Experts (MoE) models enable scalable performance but face severe memory constraints on edge devices. Existing offloading strategies struggle with I/O bottlenecks due to the dynamic, low-information nature of autoregressive expert activation. In this paper, we propose to repurpose Speculative Decoding (SD) not merely as a compute accelerator, but as an informative lookahead sensor for memory management, supported by our theoretical and empirical analyses. Hence, we introduce MoE-SpAc, an MoE inference framework that integrates a Speculative Utility Estimator to track expert demand, a Heterogeneous Workload Balancer to dynamically partition computation via online integer optimization, and an Asynchronous Execution Engine to unify the prefetching and eviction in the same utility space. Extensive experiments on seven benchmarks demonstrate that MoE-SpAc achieves a 42% improvement in TPS over the SOTA SD-based baseline, and an average 4.04x speedup over all standard baselines. Code is available at this https URL .
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.09983 [cs.LG] |
| (or arXiv:2603.09983v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.09983 arXiv-issued DOI via DataCite |
Submission history
From: Shuhuai Li [view email]
[v1]
Thu, 12 Feb 2026 04:28:00 UTC (3,135 KB)
[v2]
Fri, 23 Oct 2026 17:17:37 UTC (3,852 KB)
来源：arXiv cs.CL · arxiv.org