arXiv cs.CL· Weronika {\L}ajewska, Paul Missault, George Davidson, Saab Mansour·· 12 小时前AI 评分31
RADIUS：面向调查模拟的排序、分布与显著性综合对齐评测套件
RADIUS: Ranking, Distribution, and Significance - A Comprehensive Alignment Suite for Survey Simulation
AI 导读
RADIUS 是一套用于评估 LLM 调查模拟的两维对齐评测套件，同时考察排序对齐与分布对齐，并附带统计显著性检验。实验显示，单一指标评测会掩盖模拟器的决策相关错误：在独立复现的模拟器上，分布对齐改善往往伴随对人类最偏好选项的误判。RADIUS 已开源，以支持可复现、可比较的调查模拟器评估。
正文 · 原文
Abstract:Simulating survey responses with LLMs offers a scalable alternative to costly human data collection, yet evaluation methods remain ad hoc and inconsistent. Current metrics, typically borrowed from adjacent domains, focus on accuracy or distributional similarity while overlooking ranking alignment-whether simulations preserve the relative preference orderings observed in human populations. A simulator may achieve low distributional divergence yet fail to identify the option humans most prefer, a critical gap for decision-making applications. We propose RADIUS, a two-dimensional alignment suite that evaluates survey simulation along complementary axes: 1) RA alignment and 2) DIstribUtion alignment, accompanied by statistical Significance testing. Experiments across multiple social surveys and persona sources demonstrate that RADIUS exposes limitations of single-metric evaluation and offers stronger discriminative power. On an independently reproduced simulator, we further show that distributional improvement often coincides with misidentifying the option humans most prefer - a decision-relevant error RADIUS surfaces but distributional metrics miss. We open-source RADIUS to support reproducible and comparable assessment of survey simulators.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.19002 [cs.CL] |
| (or arXiv:2603.19002v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.19002 arXiv-issued DOI via DataCite |
Submission history
From: Weronika Łajewska PhD [view email]
[v1]
Thu, 19 Mar 2026 15:06:12 UTC (107 KB)
[v2]
Wed, 4 Nov 2026 11:39:28 UTC (294 KB)
来源：arXiv cs.CL · arxiv.org