arXiv cs.CL· Maja Pavlovic, Silviu Paun, Massimo Poesio·· 11 小时前AI 评分31
人类软标签 vs 模型不确定性：软标签学习与校准的受控审计
An Assessment of Human vs. Model Uncertainty in Soft-Label Learning and Calibration
AI 导读
研究在 MNIST 及其合成变体上对软标签学习做受控审计，通过重新标注子集提取人类不确定性，将软标签监督与标签模式偏移解耦。结果显示人类软标签虽带来准确率提升，但更大价值在于充当正则化器，改善困难样本上的模型校准并促进训练稳定收敛。数据集地图分析表明，用人类软标签训练的模型能反映人类不确定性，而用合成标签训练的模型无法与人类对齐。
正文 · 原文
Abstract:Central to human-aligned AI is understanding the benefits of human-elicited labels over synthetic alternatives. While human soft-labels improve calibration by capturing uncertainty, prior studies conflate these benefits with the implicit correction of mislabeled data (mode shifts), obscuring true effects of soft-labels. We present a controlled audit of soft-label learning across MNIST and a synthetic variant, re-annotating subsets to extract human uncertainty. By decoupling soft-label supervision from underlying label mode shifts, we show that while human soft-labels do provide accuracy gains, their larger value lies in acting as a regularizer that improves model calibration on difficult samples and promotes stable convergence across training runs. Dataset cartography reveals models trained on human soft-labels mirror human uncertainty, whereas those trained on synthetic labels fail to align with humans. Broadly, this work provides a diagnostic testbed for human-AI uncertainty alignment.
| Comments: | NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.18648 [cs.LG] |
| (or arXiv:2605.18648v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.18648 arXiv-issued DOI via DataCite |
Submission history
From: Maja Pavlovic [view email]
[v1]
Mon, 18 May 2026 16:55:21 UTC (27,207 KB)
[v2]
Mon, 5 Oct 2026 15:31:39 UTC (37,205 KB)
[v3]
Thu, 5 Nov 2026 08:34:40 UTC (37,205 KB)
来源：arXiv cs.CL · arxiv.org