arXiv cs.CL· Debarpan Bhattacharya, Malay Phadke, Sriram Ganapathy·· 11 小时前AI 评分26
BiG-SURE：用二分图估计 LLM 语义不确定性与可靠性
BiG-SURE - Bipartite Graph for Semantic Uncertainty and Reliability Estimation of LLMs
AI 导读
BiG-SURE 是一种面向黑盒 LLM 与 VLM 的无监督不确定性估计方法，通过跨温度语义一致性构建锚点-探针二分图，并以归一化平方谱能量定义置信度。在文本 QA、多语言 QA 和多模态 QA 任务上，该方法相较此前的黑盒不确定性估计器提升了平均弃答 AUROC。该工作发表于 EMNLP 2026 主会口头报告。
正文 · 原文
Abstract:Reliable uncertainty estimation is a crucial requirement for deploying large language models (LLMs) and vision-language models (VLMs) in safety-critical settings, especially when the model parameters are not accessible (black-box). We propose BiG-SURE, an uncertainty estimator based on cross-temperature semantic agreement. The method samples low-temperature responses as stable semantic anchors and high-temperature responses as probes under meaning-preserving input transformations. It then constructs an anchor-probe Bipartite Graph (BiG) using NLI-based entailment scores and defines confidence through the normalized squared spectral energy of this matrix, with uncertainty given by its complement. This bipartite graph-based Semantic Uncertainty and Reliability Estimation (SURE) score measures whether high-temperature probes remain semantically aligned with the model's stable low-temperature belief or not. We evaluate BiG-SURE on text QA, multilingual QA, and multimodal QA tasks across multiple model families. In these experiments, BiG-SURE improves average abstention AUROC over prior black-box uncertainty estimators, while remaining simple, unsupervised, and applicable to black-box model settings.
| Comments: | 22 pages, 9 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.30646 [cs.CL] |
| (or arXiv:2608.30646v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.30646 arXiv-issued DOI via DataCite |
|
| Journal reference: | EMNLP 2026 Main Conference (Oral presentation) |
Submission history
From: Debarpan Bhattacharya [view email]
[v1]
Mon, 31 Aug 2026 11:48:34 UTC (10,221 KB)
[v2]
Tue, 1 Sep 2026 09:20:59 UTC (6,785 KB)
[v3]
Fri, 6 Nov 2026 10:57:23 UTC (6,785 KB)
来源：arXiv cs.CL · arxiv.org