从信号到迁移：大语言模型中基于探针的不确定性估计的因子化研究

View PDF HTML (experimental)

Abstract:Probe-based uncertainty estimation (UE) has emerged as a prominent approach to detect hallucinations in Large Language Models (LLMs) by learning uncertainty from internal model signals. Yet, recent methods vary simultaneously across feature design, training data construction, and evaluation setting, obscuring what actually drives performance. To address this issue, we propose a factorised study of probe-based UE under matched conditions. Our results show that raw hidden states and attention features are difficult to outperform in-domain. However, under distribution shift, structured and compressed features are more robust, suggesting that in-domain performance alone is insufficient to measure progress. Furthermore, prompting and label construction significantly affect probe behaviour. Building on these empirical design principles, we construct benchmark-pretrained probes that retain useful discriminative signal on open-ended factual generation. Their competitive performance relative to limited-label target setting probes and sampling-based estimators suggests that they can provide a practical starting point for subsequent target-domain adaptation. More broadly, our work encourages deployment-oriented evaluation of probe-based uncertainty estimators.

Submission history

From: Ponhvoan Srey [view email]
[v1] Fri, 26 Jun 2026 03:23:15 UTC (1,151 KB)
[v2] Thu, 15 Oct 2026 04:02:29 UTC (2,310 KB)