arXiv cs.AI· Nektarios Kalampalikis, Pablo M. Olmos, Isabel Valera·· 6 天前AI 评分22
一次到位：联合学习准确且校准的神经网络
Calibration in One Go: Jointly Learning Accurate and Calibrated Neural Networks
AI 导读
研究者提出一种双层 Stackelberg 公式，让标准的事后校准器与主干网络端到端联合训练，无需单独的校准集或专用校准损失，仅新增校准器学习率一个超参数。该框架支持从温度缩放、矩阵缩放直到 NN 层的各类校准器，结果显示直接以最终分类层作为校准器即可获得准确且校准良好的模型，且不增加额外参数。方法以极小复杂度超越 SOTA 校准器，并避免多任务学习等场景中独立校准组件的成倍开销。
正文 · 原文
Abstract:Neural networks (NNs), while highly flexible and versatile, are often poorly calibrated. Post-hoc calibration methods adjust the outputs of an already trained model using a separate calibration dataset and additional calibration parameters. This process adds overhead that requires separate optimization of the calibrator and subsequent model selection. In this paper, we show that standard (post-hoc) calibrators can be trained end-to-end with the backbone via a bilevel Stackelberg formulation. This formulation eliminates the need for a separate calibration set or a dedicated calibration loss by jointly optimizing accuracy and calibration using cross-entropy, introducing only one additional hyperparameter: the calibrator learning rate. Although our framework supports calibrators ranging from temperature and matrix scaling to NN layers, our results show that using the final classification layer itself as the calibrator yields accurate, well-calibrated models with no additional parameters. Our approach outperforms SOTA calibrators with minimal added complexity, while also avoiding the multiplicative overhead of separate calibration components in settings such as multi-task learning.
| Comments: | 27 pages, 7 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.13693 [cs.LG] |
| (or arXiv:2610.13693v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13693 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Nektarios Kalampalikis [view email]
[v1]
Fri, 9 Oct 2026 14:54:35 UTC (799 KB)
来源：arXiv cs.AI · arxiv.org