迈向 AI 流行病学:面向前瞻性风险检测的测量标准化框架
Toward AI epidemiology: a measurement standardization framework for prospective risk detection
一篇发表于 AI & Society 的概念论文提出测量标准化框架,将专家与 AI 的交互压缩为结构化、可比较的字段,无需访问模型内部即可做前瞻性风险检测。框架定义了 8 个交互字段的语法,并给出基于配对 bootstrap 推断、DeLong 检验、0.05 单侧非劣效边界和 Holm-Bonferroni 校正的统计协议。
Abstract:This paper proposes a measurement standardization framework that compresses expert-AI interactions into structured, comparable fields for prospective risk detection in deployed AI systems, without access to model internals. This concept paper defines the framework's scope, semantically and statistically, and specifies a protocol for its empirical testing. The population-level claims it is designed to support therefore belong to a staged research program rather than to results claimed here. Measurement standardization underpins three claims. The first is a reliability claim: under bounded conditions, large language models can produce reliable, standardized assessments of the evidential and policy alignment of expert-AI interactions. The second is a governance claim: alignment scores give experts an immediate signal during deployment and give institutions a basis for monitoring alignment patterns across mission types, models, and domains. The third is an outcome validation claim: once measurement standardization is established, aggregate alignment scores could be used to study associations with downstream outcomes in regulated professional settings. This introduces the possibility of an "AI epidemiology", a form of risk detection based on correlated variables instead of mechanistic analysis, inspired by epidemiological reasoning. A minimal application of the protocol to a published expert-AI corpus shows that the judge reproduces its policy and evidential alignment scores across two runs under the specified conditions. Judge reliability at scale remains to be validated in future work. The paper sets out a defined grammar of eight interaction fields, together with a statistical protocol based on paired bootstrap inference, DeLong's test for paired AUCs as a sensitivity check, a pre-specified one-sided non-inferiority margin of 0.05, and Holm-Bonferroni correction.
| Comments: | 29 pages, 3 figures, 4 tables. Published in AI & Society (2026) |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.15783 [cs.AI] |
| (or arXiv:2512.15783v5 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2512.15783 arXiv-issued DOI via DataCite |
|
| Journal reference: | AI & Society (2026) |
| Related DOI: | https://doi.org/10.1007/s00146-026-03276-3
DOI(s) linking to related resources |
Submission history
From: Kit Tempest-Walters PhD [view email]
[v1]
Mon, 15 Dec 2025 11:29:05 UTC (487 KB)
[v2]
Thu, 19 Feb 2026 20:27:24 UTC (455 KB)
[v3]
Thu, 4 Jun 2026 11:29:58 UTC (435 KB)
[v4]
Mon, 20 Jul 2026 11:40:43 UTC (508 KB)
[v5]
Tue, 17 Nov 2026 18:30:20 UTC (301 KB)
来源:arXiv cs.AI · arxiv.org