arXiv cs.MA· Nada Rahali, Zijia Wang, Zhi-Song Liu·· 12 小时前AI 评分26
C-MoA：为 Mixture-of-Agents 提供答案级共形校准保证
Calibration Is Not Verification: Answer-Level Conformal Guarantees for Mixture-of-Agents
AI 导读
研究者提出 C-MoA，将 Mixture-of-Agents 输出拆解为原子声明、评估智能体间语义支持度，并在整答案层面校准声明保留，从而对"答案保留任何无参考标签支持声明"的概率提供无分布假设的边际控制。
正文
Abstract:Web answer synthesis with Mixture-of-Agents (MoA) requires reliable factual support for every atomic claim, yet agreement among agents is not verification. Correlated proposers can repeat a shared misconception, and a correct specialist fact receives little support when the other proposers stay silent. We introduce C-MoA, which decomposes MoA outputs into atomic claims, scores inter-agent semantic support, and calibrates claim retention at the level of whole answers. Under exchangeability and a policy frozen before calibration, it provides distribution-free marginal control of the probability that an answer retains any claim unsupported by the reference labels. Because calibration only selects a cutoff on a fixed ranking and cannot repair shared errors, we also study CONTRA-MoA, an optional extension that adds blinded near-miss tournaments, leave-one-agent-out stability, and availability-aware fusion, and we derive a competence--discrimination law for its verifier. Across 300 splits of the FactScore biography benchmark, the mean rate of fully supported answers (Fact) exceeds all six nominal targets, but only $1.7\%$--$9.8\%$ of supported claims are retained and $30\%$--$75\%$ of answers are empty. At $\alpha=0.10$, mean agreement retains $3.1\%$ of supported claims, about $3.4$ times the $0.9\%$ retained by the best single-proposer score. On the medical Golden Dataset, it retains $37.3\%$ at a mean Fact of $0.898$, slightly below the $0.90$ target, and on NQ mean Fact reaches $0.904$. On a separate balanced FactScore cohort, memory-only counterfactual scoring is close to chance (AUC $0.522$ on 182 claims), and raw maximum fusion lowers the agreement AUC from $0.687$ to $0.652$. These results separate the statistical calibration of factuality from informative claim verification and clarify the role of each in multi-agent Web synthesis.
| Comments: | 11 pages, 3 figures; revised manuscript with updated experiments and analysis |
| Subjects: | Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2609.25959 [cs.MA] |
| (or arXiv:2609.25959v2 [cs.MA] for this version) | |
| https://doi.org/10.48550/arXiv.2609.25959 arXiv-issued DOI via DataCite |
Submission history
From: Zijia Wang [view email]
[v1]
Tue, 22 Sep 2026 10:12:32 UTC (925 KB)
[v2]
Sun, 25 Oct 2026 10:05:25 UTC (182 KB)
来源：arXiv cs.MA · arxiv.org