arXiv cs.CL· Yawei Li, Jiazheng Li, David R\"ugamer, Bernd Bischl, Yulan He, Mina Rezaei·· 13 小时前AI 评分31
S-EDL：从序列似然中提取自证据，实现 LLM 语义校准
S-EDL: Eliciting Self-Evidence from Sequence Likelihoods for Semantic Calibration of LLMs
AI 导读
S-EDL 是一种新的证据式方法，通过从 LLM 自身的序列似然中提取证据，构建针对提示词语义类的可微 Dirichlet 先验，从而原生优化模型的语义校准。该方法无需固定选项即可在可变语义类上生成校准感知的训练信号，在四个开放式基准和三个模型上显著降低了语义校准误差并提升了准确率。代码已开源。
正文 · 原文
Abstract:Calibrating large language models (LLMs) in open-ended generation is uniquely challenging because many distinct token sequences can express the same underlying meaning. Prior calibration techniques target either fixed-set classification confidence or the binary correctness of individual strings, making them inherently ill-suited for the dynamic, multi-string nature of open-ended semantic outcomes. To bridge this gap, we introduce S-EDL, a novel evidential method that natively optimizes LLMs for semantic calibration. By eliciting evidence from the sequence likelihoods of the model itself, S-EDL constructs a differentiable Dirichlet prior over prompt-specific semantic classes. Optimizing the evidential loss under this prior yields a calibration-aware training signal over variable semantic classes, completely bypassing the need for fixed multiple-choice options. Evaluations across four open-ended benchmarks and three models demonstrate that S-EDL substantially reduces semantic calibration error while improving accuracy. By aligning generative likelihoods with semantic correctness, this work establishes a framework for deploying trustworthy LLMs in complex, open-ended applications. Code is available at: this https URL.
| Comments: | 34 pages, 8 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.29832 [cs.CL] |
| (or arXiv:2610.29832v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29832 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yawei Li [view email]
[v1]
Wed, 21 Oct 2026 14:55:58 UTC (543 KB)
来源：arXiv cs.CL · arxiv.org