arXiv cs.CL· Gon\c{c}alo Gomes, Bruno Martins, Chrysoula Zerva·· 11 小时前AI 评分18
BEiTScore++：基于 BEiT-3 与保形分位数回归的无参考图像描述评估
BEiTScore++: Reference-free Image Captioning Evaluation with BEiT-3 and Conformalized Quantile Regression
AI 导读
BEiTScore++ 在 BEiTScore 图像描述评估指标基础上引入不确定性感知扩展，通过建模人类评分的预测分布，在单次前向传播中同时输出对齐分数和经保形校准的置信区间。
正文 · 原文
Abstract:Reliable uncertainty estimation is important for deploying evaluation metrics in real-world settings, where both input ambiguity and domain shifts are common. In this work, we propose an uncertainty-aware extension of the BEiTScore image captioning metric, producing conformally calibrated confidence intervals alongside alignment scores. Our approach models predictive distributions over human judgments, enabling both accurate scoring and principled uncertainty quantification in a single forward pass. We evaluate our BEiTScore++ metric across multiple benchmarks with human-annotated ratings, showing that it preserves the ranking correlation of BEiTScore, while producing intervals whose width tracks the prediction error more closely than Monte Carlo dropout on most benchmarks. The estimated uncertainty also grows under image degradation and on out-of-domain artwork images, making BEiTScore++ a practical tool for uncertainty-aware evaluation of image-caption alignment.
| Comments: | Accepted to Findings of EACL 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.09505 [cs.CV] |
| (or arXiv:2611.09505v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.09505 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Findings of the Association for Computational Linguistics: EACL 2026 |
| Related DOI: | https://doi.org/10.18653/2026.findings-emnlp.2160
DOI(s) linking to related resources |
Submission history
From: Gonçalo Emanuel Cavaco Gomes [view email]
[v1]
Fri, 6 Nov 2026 14:23:34 UTC (27,160 KB)
来源：arXiv cs.CL · arxiv.org