arXiv cs.CY· Jinchang Zhang, Kaijun Jiang, Guoyu Lu·· 6 天前AI 评分27
LLM 如何利用证据线索？一项 500 场景基准研究
What Counts as Evidence? Epistemic Cue Utilization in Large Language Models
AI 导读
一项 500 场景的受控基准研究考察了 13 个 LLM 对来源、时间与呈现线索的利用情况，发现模型对来源可靠性的识别几乎一致，但当来源历史被替换时，其选择影响从可忽略到完全反转不等。对三个 Qwen 检查点的干预显示，事实重述提升了两个 27B 模型的可靠性敏感度，叠加额外加权原则后分别再提升 39.2 和 47.2 个百分点，而 9B 模型仅改变了对证据来源的敏感度。
正文 · 原文
Abstract:Which information actually shapes a language model's judgment when sources disagree? We study epistemic cue utilization through a controlled benchmark of 500 scenarios and five complementary tracks separating cue identification, cue influence, and competition between source, temporal, and presentation cues. Across 13 LLMs, nearly identical recognition of relative source reliability can accompany effects ranging from negligible to complete choice reversal when source histories are exchanged. The resulting profiles distinguish models that rely on source history from those more responsive to evidence lineage, event time, or presentation order. We then intervene on three Qwen checkpoints using matched sham material, fact restatement, structured evidence records, and an additional weighting principle. Restatement increases reliability sensitivity in the two 27B models, and adding the principle increases it by a further 39.2 and 47.2 percentage points relative to the ledger. The 9B model instead changes its sensitivity to evidence lineage without a comparable reliability response. Verified-state accuracy remains 100\% throughout these interventions. Together, the benchmark and interventions show that identifying an epistemic cue and allowing it to influence judgment are distinct, experimentally measurable aspects of model behavior.
| Subjects: | Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.12788 [cs.CY] |
| (or arXiv:2610.12788v1 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12788 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jinchang Zhang [view email]
[v1]
Thu, 8 Oct 2026 22:45:26 UTC (549 KB)
来源：arXiv cs.CY · arxiv.org