arXiv cs.CL· An-Sheng Jhang·· 11 小时前AI 评分31
多估计引导并不能统一消除语言模型判断中的偏差
Multi-Estimate Elicitation Does Not Uniformly Debias Language Model Judgments
AI 导读
研究在锚定、后见之明、基础比率、合取和现状/默认五类任务上对比直接数值回答与多估计回答格式，评估了 gpt-6.1-sol、claude-sonnet-5.5 和 Qwen3-8B（关闭推理）三种配置。23 项预注册验证测试中仅 6 项在 Holm 校正后显著，且这 6 项绝对点估计均低于预设最小效应量，未能确立等效性。多估计引导未能在任务与模型间实现统一去偏或响应衰减。
正文 · 原文
Abstract:Does eliciting five weighted estimates within a single response change how a language model responds to information? Across anchoring, hindsight, base-rate, conjunction, and status-quo/default tasks, we compare direct numerical answers with a multi-estimate response format, contrasting source framings, explicitly random cues, and probability-consistency constraints. We evaluate three completed configurations--gpt-6.1-sol, claude-sonnet-5.5, and Qwen3-8B (reasoning disabled, analyzed descriptively). In the first two configurations, direct elicitation yielded near-zero irrelevant-anchor indices and status-quo effects, while remaining responsive to informative sources and informed advice. Of 23 preregistered confirmatory tests, six rejected after Holm adjustment; however, all six absolute point estimates fell below their prespecified smallest effect sizes of interest without establishing equivalence. Moreover, while separate versus joint elicitation shifted conjunction judgments, interpretation is constrained by between-call noise. Ultimately, multi-estimate elicitation failed to produce uniform debiasing or response attenuation across tasks and models, though post hoc correlations between candidate spread and anchoring error motivate independent predictive validation. Materials: this https URL
| Comments: | 47 pages, 3 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.00597 [cs.CL] |
| (or arXiv:2611.00597v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00597 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: An-Sheng Jhang [view email]
[v1]
Sat, 31 Oct 2026 03:46:00 UTC (2,777 KB)
来源：arXiv cs.CL · arxiv.org