指令微调语言模型无法从其能描述的分布中采样

View PDF HTML (experimental)

Abstract:Silicon sampling uses language models as proxies for human survey respondents, treating each model call as an independent draw from the persona's response distribution. We show this draw does not exist: instruction-tuned models do not sample from distributions, they collapse to a single output. The same persona on the same question returns the same answer on more than half of items in a public-opinion benchmark, and the model's internal probabilities concentrate on a single option. The failure is associated with, and amplified by, instruction-targeted post-training: instruction-tuned models are worse than their own bases in every family we can compare, the gap widens at each successive post-training stage and with the size of the tuning update, and continued pretraining on non-instruction tokens leaves it unchanged. Yet the knowledge survives: the same model that cannot sample from a distribution can describe it accurately in a single call. We call this gap the KNOWS/DOES split. Exploiting the split, a single call that asks the model to describe the response distribution more than halves the error against human survey data compared to persona aggregation. When per-persona outputs are required, we propose Prompt-Perturbed Argyle (PPA), which reduces the same error by 21%, spreading each persona's answers to mirror real population differences at no added cost.

Comments:	EMNLP Findings 2026
Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2607.25292 [cs.AI]
	(or arXiv:2607.25292v3 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2607.25292 arXiv-issued DOI via DataCite

Submission history

From: Chaemin Jang [view email]
[v1] Tue, 28 Jul 2026 04:58:46 UTC (2,453 KB)
[v2] Mon, 31 Aug 2026 12:40:39 UTC (2,459 KB)
[v3] Sat, 10 Oct 2026 21:12:04 UTC (2,459 KB)