arXiv cs.AI· Andrew Semenov, Svyatoslav Dorofeev·· 10 小时前AI 评分49
Kierston & KierstonBench：Allentown, PA 城市级模拟与语言模型犯罪调查及监控任务评测
Kierston & KierstonBench: Simulation of Allentown, PA and Evaluation of Language Models in Crime Investigation and Surveillance Tasks
AI 导读
研究者发布城市级模拟引擎 Kierston 与基准 KierstonBench，模拟宾夕法尼亚州 Allentown 约 125,000 名居民的日常活动及电信、金融、行政和车牌记录，用于评测语言模型在犯罪调查与监控任务中的能力。
正文 · 原文
Abstract:Large language models are candidates for collecting and analyzing records in criminal investigations and mass surveillance. Their capability and accuracy in this work cannot currently be measured. Real deployments lack the ground truth needed to score a model's conclusions, and no benchmark evaluates these capabilities in a simulated environment. We present Kierston, a city-scale simulation engine, and KierstonBench, a benchmark that tests these capabilities against known simulated events. Kierston models approximately 125,000 residents of Allentown, Pennsylvania, generating their daily activities and the telecommunications, financial, administrative and license-plate records that surveillance and investigative systems collect about them. These records derive from a common behavioral timeline and are distributed across sources representing separate providers. Models investigate through a restricted records interface, linking observations across sources without access to the underlying simulation. The benchmark comprises 32 task families covering record linkage, routine reconstruction, incident investigation and assessment of deceptive accounts. In an evaluation of sixteen language models, most could not perform records investigation and submitted identifications the records did not support. A small number of frontier models solved most of the tasks and abstained when evidence was insufficient. In some cases these models still fell for common deceptions and assumptions, and none replaced an investigator.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.04785 [cs.AI] |
| (or arXiv:2611.04785v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.04785 arXiv-issued DOI via DataCite |
Submission history
From: Andrew Semenov [view email]
[v1]
Tue, 3 Nov 2026 12:23:08 UTC (11,828 KB)
[v2]
Tue, 10 Nov 2026 11:27:46 UTC (11,827 KB)
来源：arXiv cs.AI · arxiv.org