arXiv cs.CL· Esko Lehtonen, Tommi Buder-Gr\"ondahl·· 13 小时前AI 评分22
研究：LLM 作为问卷受访者时，预测能力会随已知回答减少而退化至简单语义基线水平
LLMs as survey respondents: Comparison to simple semantic baselines
AI 导读
研究评估了三个 LLM 在两项关于自动驾驶与轻型电动车态度的问卷中预测留出题项回答的能力，并与基于嵌入向量相似度的两个简单基线对比。LLM 多数情况下预测优于基线，但已知回答比例下降时表现持续退化，在其中一项问卷中甚至与基线持平。结果表明 LLM 的问卷回答增强能力因调查而异，数据较少时会退回到简单嵌入相似度基线的水平。
正文 · 原文
Abstract:Large language models (LLMs) have been explored as surrogates for human survey respondents. A more promising application is data augmentation, where LLMs predict new responses from partial human data. However, the semantic similarity between questionnaire items is a known source of correlation between answers. It is thus unclear whether LLMs actually infer meaningful relationships between question-response pairs, or use surface-level comparison between questions as a heuristic. To assess this, we evaluated how three LLMs can predict responses to held-out items in two surveys measuring attitudes toward automated or light-electric vehicles, based on varying proportions of known responses. Performance was compared to two baselines, where new responses were derived by maximizing embedding-similarity to a prior response. While LLMs mostly predicted human responses better than baselines, their performance consistently declined with a lower share of known responses, even aligning with the baseline in one survey. Our results show that LLMs' capacities for augmenting replies varied across surveys, and can revert to the same levels as simpler embedding similarity baselines when less data is available.
| Comments: | 16 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.29645 [cs.CL] |
| (or arXiv:2610.29645v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29645 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tommi Buder-Gröndahl [view email]
[v1]
Wed, 21 Oct 2026 12:26:20 UTC (107 KB)
来源：arXiv cs.CL · arxiv.org