arXiv cs.CL· Jaeyun Kim, Hoonki Lee, Susie Shin, Hyunmin Kim, Dahyun Lee, Sangyeob Kim·· 2 天前AI 评分22
DPA-MoE：面向 MoE 推理的运行时精度控制框架
DPA-MoE: A Runtime Precision Control Framework for MoE Inference
AI 导读
DPA-MoE 是一个让路由专家支持任意精度的运行时精度控制框架，通过 SharedScale AnyBCQ 将专家前馈权重编码为可前缀解码的 chunk，并利用实时 router gate 分数在推理时分配 chunk，使单个编码后的 MoE checkpoint 无需重训练或重新编码即可在多种有效权重预算下运行。
正文 · 原文
Abstract:Mixture-of-Experts (MoE) models reduce computation by routing each token to a small set of experts, but each selected expert is still used at a fixed precision. We present DPA-MoE, a runtime precision control framework that makes routed experts any-precision. DPA-MoE encodes expert feed-forward weights as prefix-decodable chunks using SharedScale AnyBCQ, profiles per-expert reconstruction residuals offline, and allocates chunks at inference time using live router gate scores. This post-routing allocation lets a single encoded MoE checkpoint operate at multiple effective weight budgets without retraining or re-encoding. Experiments on Qwen3-30B-A3B and DeepSeekMoE-16B show that DPA-MoE improves KL divergence and downstream accuracy over uniform chunk allocation at matched budgets, with modest steady-state throughput overhead in our prototype.
| Comments: | 18 pages, 7 figures; Accepted to Findings of EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21378 [cs.CL] |
| (or arXiv:2610.21378v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21378 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sangyeob Kim [view email]
[v1]
Thu, 15 Oct 2026 06:58:00 UTC (1,232 KB)
来源：arXiv cs.CL · arxiv.org