arXiv cs.CL· Ponhvoan Srey, Jason Lucas, Pittawat Taveekitworachai, Junyoung Lee, Anh Tuan Luu·· 13 小时前AI 评分31
用语言模型似然校准事实证据：对比式事实证据 CFE 与对比式事实后验 CFP
Truth in Contrast: Calibrated Factual Evidence from Language Model Likelihoods
AI 导读
研究提出对比式事实证据（CFE）及其概率形式对比式事实后验（CFP），通过比较同一主张在事实与非事实条件下的似然来估计 LLM 对事实陈述的置信度。分析表明 CFP 的校准取决于两个替代条件分布之间的差分误差，共享误差可相互抵消从而获得更紧的校准界；实验中 CFP 显著改善校准，事实判别力与最强基线相当。
正文 · 原文
Abstract:Large Language Models (LLMs) encode substantial factual knowledge. Yet, estimating confidence in factual claims reliably remains challenging. We study likelihood-based factual confidence from the complementary perspectives of discrimination and calibration across multilingual factual statements. Specifically, we instantiate contrastive generative likelihoods as Contrastive Factual Evidence (CFE), which compares the likelihood of the same claim under competing factual and non-factual conditions, and derive its probabilistic counterpart, the Contrastive Factual Posterior (CFP). We show analytically that CFP calibration depends on the differential error between the two surrogate conditionals, allowing errors shared across them to cancel and yield tighter calibration bounds. Empirically, CFP substantially improves calibration while maintaining factual discrimination comparable to its strongest competitor. Nonetheless, discrimination is uniformly weak across methods in lower-resource languages, which simultaneously undermines the utility of the measured calibration. To this end, we interpret a likelihood geometry, within which an evidence-bearing direction is induced, along with an orthogonal evidence-invariant direction. In our investigation, we selectively align target-language claims with stronger English evidence only along the factual-evidence direction, improving discrimination and open-ended factual generation without target-language labels.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.23527 [cs.CL] |
| (or arXiv:2610.23527v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23527 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ponhvoan Srey [view email]
[v1]
Fri, 16 Oct 2026 14:27:04 UTC (343 KB)
来源：arXiv cs.CL · arxiv.org