arXiv cs.CL· Dezhou Shen·· 12 小时前AI 评分31
解锁 Turing Tensor Cores：单张 22G RTX 2080 Ti 上从零预训练的实测吞吐报告
Unlocking Turing Tensor Cores: A Measured Throughput Account of From-Scratch Pre-training on a Single 22G RTX 2080 Ti
AI 导读
研究用 turing-unlock 内核模块在 nvidia 驱动绑定前写入 FP16 速度选择器并释放 ACR mutex，使 cuBLAS FP16 matmul 吞吐从 55.7 提升至 91.4 TFLOPS（约 1.6 倍）。
正文 · 原文
Abstract:The RTX 2080 Ti (TU102) ships full second-generation Tensor Cores in hardware, yet the vendor throttles FP16 matrix multiplication at the driver level, leaving usable tensor throughput well below the hardware peak. turing-unlock lifts this restriction with a kernel module that, before the nvidia driver binds the device, writes the FP16 speed selectors (SS0=0x88888888, SS1=0x8) and releases the ACR mutex; published measurements report cuBLAS FP16 matmul throughput rising from 55.7 to 91.4 TFLOPS (about 1.6x). On the same 22G 2080 Ti and under the same from-scratch pre-training protocol as the companion memory ledger (five tiers of Chinese encoder-decoder models, 2,451 steps per tier, 16,384 tokens per step, seed 42), we obtain a before/after throughput account: the small tier (75.7M parameters, 512-wide, 8+8 layers, activation checkpointing on) improves in steady-state throughput from 6,480 to 7,246 tok/s (+11.8%) with wall time cut from 1.77h to 1.61h (9.0% saved); the tiny tier goes from 15,150 to 15,866 tok/s (+4.7%) with wall time savings of 3.8%; the three smaller tiers (pico-micro) gain at most +0.3%; per-tier peak memory stays bit-identical, and final quality metrics are bit-identical or consistent within 10^-3. We conclude that the unlock gain is a function of matrix size, rising monotonically with tier -- imperceptible for pico-micro, visible from tiny upward, and cashed out as a double-digit percentage at small -- and that it is a pure throughput gain that leaves training dynamics untouched.
| Comments: | 7 pages, 4 tables |
| Subjects: | Hardware Architecture (cs.AR); Computation and Language (cs.CL); Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF) |
| ACM classes: | C.4; I.2.1; I.2.6 |
| Cite as: | arXiv:2611.05419 [cs.AR] |
| (or arXiv:2611.05419v1 [cs.AR] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05419 arXiv-issued DOI via DataCite |
Submission history
From: Dezhou Shen [view email]
[v1]
Mon, 2 Nov 2026 02:51:56 UTC (285 KB)
来源：arXiv cs.CL · arxiv.org