arXiv cs.CL· Chenyang Yu, Zifan Zeng, Qunli Zhang, Feng Liu·· 12 小时前AI 评分22
STEP：通过逐步概率评分软估计预测正确性的推理轨迹
STEP: Soft Trajectory Estimation of Prediction Correctness via Per-Step Probability Scoring
AI 导读
研究者提出 STEP，一个无需训练的框架，通过轻量前向传播为每个推理步骤打分，得到候选答案上的软概率分布 qi，并据此提取刻画逐步置信度及其向最终答案演进的软轨迹特征。
正文
Abstract:Estimating the correctness of large language model (LLM) predictions without access to ground truth is critical for reliable deployment in high-stakes settings. Existing approaches, such as entropy and perplexity, operate solely on final output statistics, discarding the rich information encoded in intermediate reasoning steps. To address this limitation, we propose STEP (Soft Trajectory Estimation of Prediction correctness), a training-free framework that scores each reasoning step via lightweight forward passes to obtain a soft probability distribution qi over candidate answers, from which we derive a set of soft trajectory features capturing per-step confidence and its progression toward the final answer. Extensive experiments on various benchmarks demonstrate consistent and substantial improvements, particularly in AUROC, over output-based baselines and hard-indicator trajectory methods. Notably, Temporally-Weighted Consistency, which assigns higher importance to later reasoning steps where answer distributions have stabilized, contributes the largest gains, highlighting the importance of temporal structure in reasoning trajectories. These results further validate the advantage of preserving continuous probability signals over binary discretization.
| Comments: | Accepted at the Third Workshop on Uncertainty-Aware NLP (UncertaiNLP 2026) at EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.34537 [cs.CL] |
| (or arXiv:2610.34537v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34537 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zifan Zeng [view email]
[v1]
Sun, 25 Oct 2026 19:57:59 UTC (611 KB)
来源：arXiv cs.CL · arxiv.org