arXiv cs.CY· Yusuke Kumagae, Masahiro Kaneko·· 13 小时前AI 评分39
研究提出受访者覆盖缺口 RCG：六款 LLM 的硅采样答案分布吻合但受访者不匹配
Silicon Samples Match the Answers but Not the Respondents
AI 导读
研究提出受访者覆盖缺口（RCG）指标，以完整答案画像衡量人类受访者与 LLM 生成受访者之间的距离。在两份美国社会调查上，评估的六款 LLM 均未覆盖人类受访者，主要设置下表现不及按人类边际分布独立采样各答案的基线。所有测试的提示与推理设置中，每款 LLM 的 RCG 都高于拟合人类数据的高斯 copula。
正文 · 原文
Abstract:Silicon sampling generates survey respondents by having a large language model (LLM) answer a survey as a person with given demographic attributes, and the resulting synthetic populations are increasingly used in place of human ones. Existing evaluations focus on individual-answer accuracy, per-question answer distributions, and correlations across questions. None of these compares the respondents themselves, so a generator whose respondents have combinations of answers found in no human respondent can still score well if it reproduces every answer distribution. We propose the respondent coverage gap (RCG), which represents each respondent as a point defined by their complete answer profile and measures how far each human respondent is from the nearest generated one, relative to the distance between human samples. On two US social surveys, all six LLMs we evaluate leave human respondents uncovered, and under our main settings, they fall behind a baseline that samples each answer independently from the human marginals. Across the prompting and inference settings we test, every LLM has a higher RCG than a Gaussian copula fitted to the human data.
| Subjects: | Computation and Language (cs.CL); Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.30900 [cs.CL] |
| (or arXiv:2610.30900v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30900 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yusuke Kumagae [view email]
[v1]
Thu, 22 Oct 2026 08:22:50 UTC (193 KB)
来源：arXiv cs.CY · arxiv.org