arXiv cs.AI· Bin Wang, Yan Zhong, Zhengyu Zhang, Liang Luo, Buyun Zhang, Ellie Wen·· 13 小时前AI 评分34
NxN E-valuation：用共形 CRT 零假设认证语言模型提出的假设
NxN E-valuation: Hypothesis Certification via a Conformal CRT Null
AI 导读
研究者提出 NxN e-valuation 方法，只需假设本身即可对语言模型生成的假设进行认证，满足逐假设自动化、可归因、可诊断三项要求。在 285 个语言模型撰写、人类专家标注的真实购买数据假设上，该方法 693 次决策中 692 次正确、仅 1 次错误认证，优于需逐假设重采样模型的 CRT 组合测试（659 次正确、4 次错误）和留出验证（224 次正确、468 次错误）。
正文 · 原文
Abstract:Language models now propose hypotheses faster than anyone can check them, so the step that certifies them must need nothing from the proposer but the hypothesis. We state three requirements for such a step -- per-hypothesis automatable, attributable and diagnostic -- and introduce NxN e-valuation, which meets them. A hypothesis names a scope, the units it is about, and gives each of them a restriction that scores its outcome. The units of one scope serve as nulls for one another, so no resampling model is written per hypothesis; the algorithm certifies a stable, scope-level association and reports whether the scope, the restriction, or which unit receives which restriction carries the prediction. For independently drawn units, betting e-values bound the chance of a false certificate for each hypothesis in finite samples, under stated nulls. A betting lower bound on the share of units that carry the association gates every certificate, and an asymptotically valid equivalence test calls the restrictions shared only when it rejects differences between them above a declared noise ceiling. We test it on planted data, on randomized intervention data, and on real purchase data with 285 hypotheses that language models wrote without knowing our method or our classes, labeled by human experts. Meeting all three requirements costs no accuracy: NxN makes 692 correct decisions in 693 with one false certificate, against 659 with four for a composite of CRT tests that returns our verdicts but needs a resampling model per hypothesis, and 224 with 468 for held-out validation. Removing any of our statistics merges a verdict that certifies with one that rejects. The certificates concern predictive association, not causality.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.06621 [cs.AI] |
| (or arXiv:2608.06621v5 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.06621 arXiv-issued DOI via DataCite |
Submission history
From: Bin Wang [view email]
[v1]
Thu, 6 Aug 2026 22:14:53 UTC (50 KB)
[v2]
Sat, 29 Aug 2026 10:49:17 UTC (98 KB)
[v3]
Thu, 3 Sep 2026 22:12:40 UTC (99 KB)
[v4]
Mon, 5 Oct 2026 08:47:09 UTC (124 KB)
[v5]
Mon, 26 Oct 2026 19:19:30 UTC (121 KB)
来源：arXiv cs.AI · arxiv.org