arXiv cs.CL· Donghoon Han, Minseop Jung, Dong Woo Kwak, SeungJae Lee, Junhoo Lee·· 2 天前AI 评分27
大语言模型查找表的两比特量化：Conditional Program 压缩 n-gram 记忆
N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models
AI 导读
研究者提出 Conditional Program（CP），一种只重建输入所请求行的压缩表示，用短码加局部缩放并共享重建程序，以 Gaussian trellis 实例化。其两比特编码相比 2-bit round-to-nearest 将输出 KL 降低 41–63%。在 DeepSeek 上，该方案把表存储从 203GB 降至 51GB，为表常驻 GPU 时留出更大工作负载空间。
正文 · 原文
Abstract:Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. CP represents each row by a short code and a local scale, with a reconstruction program shared across the table. We instantiate this representation with a Gaussian trellis, norm restoration and independently decodable row records. CP's two-bit code lowers output KL by 41--63\% relative to 2-bit round-to-nearest. Analysis across three trained n-gram memory families shows that Gaussian-like coordinate statistics coexist with learned relationships among rows; at matched storage budgets, the shared trellis outperforms the fitted codebooks and low-rank residual variants we test. For DeepSeek, it reduces table storage from 203GB to 51GB, leaving room for larger workloads when the table is kept on GPU.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21810 [cs.LG] |
| (or arXiv:2610.21810v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21810 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Donghoon Han [view email]
[v1]
Thu, 15 Oct 2026 12:57:19 UTC (185 KB)
来源：arXiv cs.CL · arxiv.org