arXiv cs.CL· Edward Nguyen, Thai Tran, Kiet Anh Hoang, Duy Hoang, Duy Vo, Khanh Le, Vu Le·· 12 小时前AI 评分26
EDWARD-NMT：面向神经机器翻译的自适应路由与双重一致性弹性维度架构
EDWARD-NMT: Elastic Dimensions With Adaptive Routing and Dual-consistency for Neural Machine Translation
AI 导读
EDWARD-NMT 是一种面向 NMT 的 MoE 编码器-解码器架构，通过 Matryoshka 式弹性维度、带任务条件学习阈值的自适应 sigmoid 路由器和双重一致性目标，实现一次训练、多规格部署。
正文 · 原文
Abstract:Sparse Mixture-of-Experts (MoE) is a powerful paradigm for scaling Neural Machine Translation (NMT) and managing language heterogeneity. However, practical limitations hinder its deployment on resource-constrained devices: the entire expert pool must remain in memory, and the per-token compute budget is statically fixed during training by rigid top-k routing. We introduce EDWARD-NMT (Elastic Dimensions With Adaptive Routing and Dual-consistency for NMT), a novel MoE encoder-decoder architecture designed for a train once, deploy anywhere paradigm that exposes a family of deployable sub-networks at inference time. EDWARD-NMT integrates three key innovations: (i) Matryoshka-style elastic dimensions to flexibly reduce the memory footprint; (ii) an adaptive sigmoid router with a learned, task-conditioned threshold, allowing each token to dynamically determine its active expert count; and (iii) a dual-consistency objective to maximize the performance of the extracted sub-networks. Extensive experiments on PhoMT (En-Vi), WMT-14 (En-De), multilingual OPUS-16, and a multi-domain De-En benchmark validate our approach. The full-capacity architecture matches or exceeds strong MoE baselines with improved parameter efficiency. Furthermore, the sub-networks seamlessly extracted from a single checkpoint consistently outperform comparably sized dense Transformers without any retraining.
| Comments: | Accepted to EMNLP 2026 (Findings) |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.32115 [cs.CL] |
| (or arXiv:2610.32115v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32115 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kiet Anh Hoang [view email]
[v1]
Fri, 23 Oct 2026 03:33:58 UTC (1,048 KB)
来源：arXiv cs.CL · arxiv.org