arXiv cs.CL· Teerawat Tongloy, Jiraporn Pongsopa, Rangsan Jomtarak·· 11 小时前AI 评分32
基于量子测量与精确词证据的 Born Transformer：在光量子与量子比特上实现可解释注意力
Reading Why: Born-Measured Attention with Exact Word Evidence on Light and Qubits
AI 导读
研究者构建了 Born Transformer，将注意力、前馈非线性与读出全部改为 Born 测量，使每个类别的概率成为词振幅的多项式，其积分梯度证据可由有限求积精确计算，词贡献求和与类别概率的误差在 10⁻¹¹ 以内。
正文 · 原文
Abstract:When a language model answers, we can usually only estimate why. We build a transformer in which every step is a Born measurement - attention $|\langle q|k\rangle|^4$ normalised over each row, a measured product in place of the feed-forward nonlinearity, and a Born readout. With its denominators held at their values for the input, each class probability is a polynomial in the word amplitudes, so its integrated-gradient evidence is computed by a finite quadrature with no quadrature error: the words' contributions sum to the class probability above its no-information share, to $10^{-11}$ or better. On e-SNLI natural language inference, with about 98k parameters outside the embedding, the model reaches $76.98 \pm 0.17$% test accuracy, level with a softmax transformer of the same size ($76.62 \pm 0.36$%). Deleting the word its evidence ranks first changes the answer in 45% of test pairs, out of the 53% any single deletion can change, against 32% for the softmax model explained the same way and 40% by its attention x gradient. Attention can look as much like the words annotators highlighted as the evidence does, yet accounts for the answer less well: deleting the word it ranks first changes the Born Transformer's answer in 32% of pairs. The same weights compile exactly to optoelectronic optics and to qubits, and a coherent attention layer becomes one circuit equal to the model (852 CNOTs on 23 qubits). On IBM's ibm_kingston, reading only the attention overlaps by Hadamard tests, the device gave the model's answers, also when a word was deleted. With calibration circuits as deep as the overlaps, each word's evidence with the device's attention row sums followed the exact evidence (correlations 0.997 and 0.932 on two pairs chosen in advance); where a pair's evidence is a small total of large opposite terms the earlier runs say it does not, and we say why.
| Comments: | 27 pages, 8 figures, 12 tables. Code, trained models and every IBM job record: this https URL (doi:https://doi.org/10.5281/zenodo.23938333) |
| Subjects: | Quantum Physics (quant-ph); Computation and Language (cs.CL); Machine Learning (cs.LG); Optics (physics.optics) |
| Cite as: | arXiv:2611.05355 [quant-ph] |
| (or arXiv:2611.05355v2 [quant-ph] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05355 arXiv-issued DOI via DataCite |
Submission history
From: Teerawat Tongloy [view email]
[v1]
Tue, 3 Nov 2026 18:52:44 UTC (300 KB)
[v2]
Wed, 4 Nov 2026 16:21:50 UTC (346 KB)
来源：arXiv cs.CL · arxiv.org