arXiv cs.AI· Ziyad Benomar, Weronika {\L}ajewska, Leonardo Perelli, Saab Mansour·· 6 天前AI 评分32
基于数据驱动人设的 LLM 智能体用于 A/B 测试模拟
Data-Driven Persona-Conditioned Agents for A/B Test Simulation
AI 导读
研究者提出一种 A/B 测试模拟框架，用 LLM 智能体承载基于真实用户行为信号构建的数据驱动人设，替代此前的合成或规则型人设。在覆盖两类指标的 40 项 A/B 测试基准上，最佳配置取得 0.75-0.80 的方向准确率，可用于快速、低成本的实验预筛。该工作已被 EMNLP 2026 Industry Track 与 REALM Workshop 接收。
正文 · 原文
Abstract:A/B testing is the gold standard for evaluating product changes, but each experiment requires real user traffic, engineering effort, and weeks of measurement. We propose a simulation framework that predicts A/B test outcomes using LLM-powered agents conditioned on data-driven personas grounded in real user behavioral signals. Unlike prior work that relies on synthetic or rule-based personas, our agents are constructed from anonymized behavioral data - activity patterns, engagement signals, and inferred demographics - enabling more faithful population modeling. We frame A/B test simulation as a structured question task and systematically study (i) question design formats, (ii) the impact of persona data source and domain alignment, (iii) the trade-off between per-persona behavioral depth and population diversity, and (iv) efficient population subsampling. On a benchmark of 40 A/B tests spanning two metric types, our best configuration achieves 0.75-0.80 directional accuracy depending on the test metric, demonstrating that data-driven personas are a viable path toward fast, low-cost experiment pre-screening.
| Comments: | Work accepted at EMNLP 2026 Industry Track and EMNLP 2026 REALM Workshop |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.01038 [cs.AI] |
| (or arXiv:2609.01038v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.01038 arXiv-issued DOI via DataCite |
Submission history
From: Weronika Łajewska PhD [view email]
[v1]
Tue, 1 Sep 2026 10:35:49 UTC (270 KB)
[v2]
Fri, 9 Oct 2026 08:06:46 UTC (269 KB)
来源：arXiv cs.AI · arxiv.org