arXiv cs.CL· Duc-Vinh Tran·· 15 小时前AI 评分34
对话问答中的答案载荷、持续性与分离的置信信号：不止一种信念
Not One Belief: Answer Payloads, Persistence, and Dissociated Confidence Signals in Conversational Question Answering
AI 导读
研究将对话问答中的用户追问拆分为言语行为（重问、同意、质疑）与答案载荷，发现答案改变几乎只由用户提供答案字符串驱动：在全部保留的全尺寸实验格中，正确答案纠正错误答案的比例至少是重问或同意的四倍。
正文 · 原文
Abstract:Conversational QA systems should fix errors when users push back and signal which answers to trust. We split each follow-up into its speech act (re-ask, agree, or challenge) and its payload (an answer string, correct or not), and measure four confidence signals on the same transcripts: verbalized confidence, length-normalized likelihood, a frozen hidden-state probe, and P(True). Answers change when the user supplies an answer string and almost never under the speech act alone: in every retained full-size cell, the correct answer converts at least four times as many wrong answers as re-asking or agreement. The four signals disagree. On Qwen2.5-7B the frozen probe rises with answer persistence rather than truth; this holds on held-out questions, with Llama-3-8B (cleaned answers) as support, but not on four other models. Among 81 Qwen2.5-7B errors that survive one re-ask, re-asking, a fresh chat, or abstain repair almost none, while a gold answer repairs 75-95%. In two preregistered replications, every repair rate without an answer string stays below 0.15; a larger post-hoc rerun excludes 0.15 except for the empty elicit. With BM25 TriviaQA evidence, sentences that contain the answer repair 67-76% of persistent errors versus at most 12% when they do not. An end-to-end contain-gate cuts harm on TriviaQA, but a deployable gate that does not know the gold answer repairs few errors.
| Comments: | 25 pages + supplementary PDF. Preprint |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.35694 [cs.CL] |
| (or arXiv:2610.35694v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35694 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vinh Tran Duc [view email]
[v1]
Mon, 26 Oct 2026 16:23:15 UTC (60 KB)
来源：arXiv cs.CL · arxiv.org