arXiv cs.CL· Shuai Guo, Yidong Cui·· 9 天前AI 评分27
BeliefScope：诊断大语言模型中的证据驱动修正与压力诱导偏移
BeliefScope: Diagnosing Evidence-Driven Revision and Pressure-Induced Shifts in Large Language Models
AI 导读
研究团队提出 BeliefScope，一个受控黑盒框架，用于区分大语言模型对同一命题的响应变化究竟来自相关证据还是来自不添加事实的用户定向压力。该框架在 36 个 Qwen/Llama 模型家族上开展研究，并对 12 个家族（含 Gemma3-12B）做定向检查，结果显示模型层面的广泛差异在匹配控制、解码或响应接口下可能改变，而部分模型内部的窄范围模式保持稳定。
正文 · 原文
Abstract:A language model may revise the same proposition after receiving genuinely relevant evidence or after receiving directional user pressure that adds no relevant fact. The observable response shift alone therefore does not reveal which source drove the change. We introduce BeliefScope, a controlled black-box framework for separating these two sources of influence around a fixed target proposition. BeliefScope crosses Evidence and Pressure with factor-specific local controls and measures response changes through probability reports, categorical judgments, and action recommendations on channel-appropriate scales. To determine when these observable contrasts support reliable attribution, we evaluate the observation design under controlled synthetic conditions. Known-truth recovery and targeted ablations establish where Evidence- and Pressure-related effects can be separated, while semi-synthetic stress tests map how that recoverability changes as the observation process becomes noisier and more heterogeneous. Across a 36-family Qwen/Llama study, with targeted 12-family checks that also include Gemma3-12B, the resulting profiles show substantial evaluation-context dependence: broad model-level differences can change under matched controls, decoding, or response interfaces, while some narrower within-model patterns remain stable. Instruction interventions further show that reduced target-aligned Pressure following can reflect either stable resistance or movement in the opposite direction. BeliefScope summarizes these measurements as a conditional belief-response profile that keeps diagnostic effects tied to the evaluation conditions under which they are observed, together with explicit validity boundaries for that diagnosis.
| Comments: | 18 pages, 13 figures, 18 tables. Includes appendices |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.11305 [cs.CL] |
| (or arXiv:2610.11305v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11305 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shuai Guo [view email]
[v1]
Thu, 8 Oct 2026 06:11:48 UTC (1,214 KB)
来源：arXiv cs.CL · arxiv.org