arXiv cs.AI· Mohammed Nayeem Khan, Allan Kevin T·· 1 天前AI 评分31
同一模型两种精度：量化副本间的分歧能否指示小模型答错？
One Model, Two Precisions: Does Disagreement Between Quantized Copies Tell You When a Small Model Is Wrong?
AI 导读
研究在 Qwen3-0.6B 与 Qwen3-1.7B 上模拟 3 至 6 bit 分组量化，用 8-bit 与低精度副本的分歧作为逐题错误分数，评测 8,398 道 ARC-Easy、ARC-Challenge、MMLU 和 HellaSwag 选择题。
正文 · 原文
Abstract:Local language models often exist at several precisions, and quantization is known to change individual answers. This study asks whether the disagreement between two precisions of the same weights, an 8-bit copy that answers and a lower-precision copy that reads the same prompt, works as a per-question error score for abstention, escalation and re-asking. The cost is one extra forward pass over fewer bytes, with no sampling and no training. We simulate round-to-nearest quantization with per-group scales (3 to 6 bits, groups of 32 to 256) on Qwen3-0.6B and Qwen3-1.7B and score 8,398 multiple-choice questions from ARC-Easy, ARC-Challenge, MMLU and HellaSwag subsets. Error detection is measured by AUROC, risk-coverage curves and ECE on a held-out half, with paired bootstrap intervals over questions. For Qwen3-0.6B, the entropy of the 8-bit answer reaches AUROC 0.682; the best disagreement score, selected on a development half, reaches 0.684 (+0.001 [-0.009, +0.011]), and combining it with the confidence features adds +0.009 [+0.002, +0.015], mostly on ARC. On Qwen3-1.7B the selected score is -0.001 [-0.009, +0.006] from entropy and the combination +0.003 [-0.001, +0.006] on the test half. Gaussian weight noise at matched error energy is about as informative as rounding (0.6B only), so we find no evidence that the signal is specific to quantization, and shuffling the features within task and entropy cells removes the gain. One extra 8-bit pass that shifts the option order does better (+0.049 [+0.031, +0.067] over entropy) at about a third more weight traffic; routing to a larger model on free entropy beats both extra-pass scores at equal total traffic. After logistic recalibration the 0.6B confidences are calibrated within a few points of ECE. A deployment recipe and the cases where the method does not help are given.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.16478 [cs.AI] |
| (or arXiv:2611.16478v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.16478 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Allan Kevin T [view email]
[v1]
Thu, 12 Nov 2026 08:47:29 UTC (1,568 KB)
来源：arXiv cs.AI · arxiv.org