arXiv cs.CY· Tom Sheffer, Alon Miron, Asael Sklar, Yaniv Dover, Ariel Goldstein·· 13 小时前AI 评分44
知识-确定性多样性可预测人-人与人类-AI开放式讨论协同效应，并揭示纯AI讨论的不足
Knowledge-Certainty Diversity Predicts Human-Human and Human-AI Open-Ended Discussion Synergy and Reveals Pure-AI Discussion Shortfalls
AI 导读
研究对比LLM-LLM、LLM三人组、人类三人组与人-LLM配对四种设置发现，含人类的交互均产生协同效应，纯LLM组表现不升反降。团队提出知识-确定性框架，用准确率与置信度刻画参与者，量化“一方不确定时另一方倾向正确”的程度，并给出基于置信度选择答案的收益上界Potential Conversation Synergy。
正文 · 原文
Abstract:Conversations transform individual knowledge into collective insight, enabling collaborators to solve problems more accurately than they could alone. Whether dialogues among large language models (LLMs) can replicate the synergistic gains observed in human discussion remains unclear. We systematically compared four interaction settings, LLM-LLM pairs, LLM trios, human trios, and human-LLM pairs, using validated medical multiple-choice questions. Agents answered individually, engaged in open-ended discussion, then re-answered, allowing us to quantify conversational gains. Interactions that included humans consistently yielded synergy (post-discussion accuracy increased for both stronger and weaker participants), whereas purely LLM groups did not improve and often declined. To explain and anticipate when open-ended dialogue helps, we introduce an agent-agnostic knowledge-certainty framework that models each participant by performance (accuracy) and confidence. This framework quantifies knowledge-certainty diversity, the degree to which one agent tends to be correct when another is uncertain, and yields an upper bound on the gain available from choosing between the agents' answers on the basis of confidence, which we term Potential Conversation Synergy. Computed before any discussion, this metric was low in both settings where discussion did not help and high in both where it did: When knowledge-certainty diversity is low (as in LLM-only groups), discussion doesn't improve performance; when it is present (as in human or human-LLM groups), open-ended dialogue reliably lifts accuracy. These findings suggest a diagnostic for AI collaboration: knowledge-certainty diversity can be measured from pre-discussion responses and provides a configuration-level indicator of conversational gains across the settings studied.
| Subjects: | Human-Computer Interaction (cs.HC); Computers and Society (cs.CY) |
| Cite as: | arXiv:2507.22889 [cs.HC] |
| (or arXiv:2507.22889v3 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2507.22889 arXiv-issued DOI via DataCite |
Submission history
From: Tom Sheffer [view email]
[v1]
Sun, 15 Jun 2025 05:09:20 UTC (2,148 KB)
[v2]
Thu, 9 Oct 2025 16:45:21 UTC (3,248 KB)
[v3]
Wed, 21 Oct 2026 02:09:41 UTC (6,954 KB)
来源：arXiv cs.CY · arxiv.org