arXiv cs.AI· Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, Michael S. Bernstein·· 4 小时前AI 评分52
研究:基于丰富自我报告的AI模拟更准确、失真更少
AI Simulations Grounded in Rich Self-Reports Increase Accuracy and Reduce Distortion
AI 导读
研究团队开发AI语音访谈员,按American Voices Project协议访谈1052名美国成年人,并对比人口学属性、自撰人设、问卷、访谈及两者组合等提示方式下的模拟效果。
正文 · 原文
Authors:Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, Michael S. Bernstein
Abstract:AI simulations of people built from demographic prompts often misjudge individuals and distort sample-level patterns. We test whether rich self-report data reduce these problems. We developed an AI voice interviewer and conducted two-hour interviews following the American Voices Project protocol with 1,052 U.S. adults, who also completed surveys and behavioral experiments. For each participant, we compared simulations prompted with demographic attributes, self-authored personas, survey responses, interview responses, or combined surveys and interviews. We evaluated accuracy on held-out responses, and measured eight statistical distortions at the sample level: biased averages, compressed variance, distorted response distributions, over-correlated items, attitudes collapsing onto one dimension, ideological sorting, amplified experimental effects, and hyper-rationality. On General Social Survey (GSS) items, demographic simulation led to 60.4% accuracy, whereas interview and survey simulation achieved 69.4% and 69.3%, and combining them reached 72.8%, compared with 79.5% for participants' two-week test-retest agreement. Surveys and interviews also improved prediction of personality traits, but not of incentivized economic choices. Combined interview and survey simulation closed a median of 53.5% of the eight distortion gaps between demographic simulation and participant responses. An open-weight model fine-tuned on other participants' self-reports and responses to the target questions achieved 75.2% GSS accuracy and closed a median of 69.3% of the distortion gap. Overall, both frontier and fine-tuned models reduced distortions without erasing them, but neither reduced hyper-rationality in economic games. These findings demonstrate that qualitative and quantitative self-reports can improve person-specific simulations and reduce statistical distortions.
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2411.10109 [cs.AI] |
| (or arXiv:2411.10109v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2411.10109 arXiv-issued DOI via DataCite |
Submission history
From: Michael Bernstein [view email]
[v1]
Fri, 15 Nov 2024 11:14:34 UTC (2,928 KB)
[v2]
Wed, 22 Apr 2026 03:48:01 UTC (5,565 KB)
[v3]
Sun, 28 Jun 2026 22:37:51 UTC (3,480 KB)
[v4]
Mon, 16 Nov 2026 21:52:11 UTC (5,351 KB)
来源:arXiv cs.AI · arxiv.org