arXiv cs.AI· Tae-Eun Song·· 10 天前AI 评分29
跨模型审查何时有效？LLM 验证中的第二模型评审实验
When Does a Second Model Help? Cross-Model Review in LLM Verification
AI 导读
一项受控实验用 30 个工件、150 个植入错误和 900 次审查会话，测试不同模型交叉审查是否优于同模型审查。
正文 · 原文
Abstract:Large language models now generate code, documentation, and analyses, and are increasingly used to review such output. We ask when a second review by a different model helps. Building on the author's earlier preprints, which varied context, repetition, and role structure within one model, we test model independence in a controlled experiment: 30 artifacts with 150 planted errors, 10 review conditions, and 900 review sessions with three reviewer models from two developers. In this experiment, (1) a top-tier cross-model reviewer is not significantly different in F1 from same-model review in a fresh session (CCR), which does not establish equivalence; (2) the two find partly different errors (Jaccard 41.2%); and (3) at two review calls, one CCR plus one cross-model review matches more planted errors than two CCR reviews (56.7% vs. 42.7%; Holm-adjusted p=.006), but not significantly more than two reviews by the top-tier cross-model reviewer, so model difference and reviewer capability are not separated. A lightweight cross-model reviewer scores no higher than same-model review. Withholding requirements from the reviewer raises F1 for the two lower tiers but not the top tier, in untested point estimates whose pattern depends on how failed sessions are scored. Before analysis we audited all session records, excluding one baseline run of uncertain provenance and 14 failed calls; results with all sessions are also reported. A partial check on public detector outputs from another benchmark neither replicates nor contradicts the main comparison. Records, artifacts, and scripts are available from the author on request.
| Comments: | 16 pages, 2 figures, 7 tables. Follow-up to arXiv:2603.12123 and arXiv:2603.21454. v2: corrects two condition labels in Table 1 (CCR sees the artifact only; SA runs in a new session) and dependent interpretations; adds review prompts, a TP/FP breakdown by severity, and limitations; states how each reviewer was run; softens case studies. Numbers unchanged except removed B5 percentages |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Software Engineering (cs.SE) |
| Cite as: | arXiv:2610.01471 [cs.CL] |
| (or arXiv:2610.01471v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01471 arXiv-issued DOI via DataCite |
Submission history
From: Tae-Eun Song [view email]
[v1]
Thu, 1 Oct 2026 11:12:36 UTC (225 KB)
[v2]
Tue, 6 Oct 2026 01:50:15 UTC (227 KB)
来源：arXiv cs.AI · arxiv.org