arXiv cs.CL· Sebastian Pohl, Harsh Mehta, Pranav Mambayil, Abdul Ghafoor, Franziska Lesigang, Yufang Hou, Christian Hilbe·· 12 小时前AI 评分45
研究：LLM 难以在受控环境中模拟人类信念更新
LLMs struggle to simulate human belief updates in controlled environments
AI 导读
一项研究用 391 名英国 Prolific 参与者的真实数据测试六个 LLM 能否模拟个体信念更新，结果显示五个模型在给定真实初始立场时无系统性方向偏差，但个体误差差异显著。
正文 · 原文
Abstract:LLMs are increasingly deployed as proxies for human study participants in social science experiments, yet the fidelity of this practice has rarely been tested directly. We test whether six LLMs can simulate individual human belief updates, comparing LLM outputs 1-to-1 against ground truth data from 391 UK participants on Prolific, who updated their stances on three discussion topics after reading Reddit comments. Each participant was simulated by an LLM conditioned on a persona derived from their demographic and personality trait data. Given participants' actual initial stances, five of the six models showed no systematic directional bias in their simulated post-stances, although the size of their individual-level errors varied considerably between models. All six models fail to simulate initial stances themselves and to produce faithful belief updates from self-generated stances. Three systematic biases emerge across all models: overrepresentation of neutral positions, more frequent but smaller belief shifts than humans, and a failure to rank comments by convincingness. Demographic and personality trait personas had no consistent effect on fidelity. We conclude that without grounding in realistic initial stances, LLM simulations of human belief dynamics cannot be considered reliable, a condition that current multi-round social media simulations rarely meet.
| Subjects: | Computation and Language (cs.CL); Social and Information Networks (cs.SI) |
| Cite as: | arXiv:2607.28347 [cs.CL] |
| (or arXiv:2607.28347v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.28347 arXiv-issued DOI via DataCite |
Submission history
From: Sebastian Pohl [view email]
[v1]
Thu, 30 Jul 2026 15:19:18 UTC (1,369 KB)
[v2]
Mon, 2 Nov 2026 12:32:53 UTC (1,929 KB)
来源：arXiv cs.CL · arxiv.org