arXiv cs.CL· Hanxin Zhang·· 2 天前AI 评分27
TurboMamba：用词法记忆实现更快推理、更快训练与更优预测
TurboMamba: Faster Inference, Faster Training, and Better Prediction with Lexical Memory
AI 导读
TurboMamba 是一种语言模型架构，由每完成一个 token 块运行一次的全局 Mamba-3 模型与一个小型 Transformer 组成，后者结合全局上下文和块内前序 token 预测下一个 token。
正文 · 原文
Abstract:We introduce TurboMamba, a language-model architecture for faster inference, faster training, and better token prediction. A global Mamba-3 model runs once per completed token block. A small Transformer uses this global context and earlier tokens within the block to predict the next token. Three lightweight additions improve local predictions: direct access to the previous block's last token, a small correction to next-token scores, and a learned mixture with recent-token frequencies. We compare twelve small-scale implementations on public FineWeb-Edu, with comparable parameter counts and shared training-budget caps. The baselines include Transformer, Mamba-3, Jamba, and Zamba. The Jamba and Zamba versions use Mamba-3 in place of their original Mamba modules. TurboMamba achieves the lowest mean test prediction loss (negative log-likelihood, NLL) among all twelve models. Its mean NLL is 9.4% lower than that of Mamba-3, 7.3% lower than that of the Jamba version, and 18.3% lower than that of the Zamba version. End-to-end training throughput is 2.60x that of Mamba-3, 2.81x that of the Jamba version, and 4.82x that of the Zamba version. We measure end-to-end batch latency for a batch of 32 sequences, each with a 256-token prompt and 128 generated tokens. On this workload, TurboMamba achieves a 3.12x speedup over Mamba-3, a 3.53x speedup over the Jamba version, and a 6.21x speedup over the Zamba version.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20682 [cs.LG] |
| (or arXiv:2610.20682v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20682 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hanxin Zhang [view email]
[v1]
Wed, 14 Oct 2026 16:29:19 UTC (79 KB)
来源：arXiv cs.CL · arxiv.org