AutoAnalysis：自动自我更新的人机协作元分析

View PDF HTML (experimental)

Abstract:As generative artificial intelligence (AI) becomes more capable and widely used, understanding when and how humans should work with these systems is becoming increasingly important. However, the empirical evidence in this space offers conflicting recommendations, which leaves individuals and organizations without clear guidance on when to combine human and AI capabilities. Existing meta-analyses can help reconcile these findings, but they require substantial manual effort and cannot readily incorporate new evidence as it emerges. To address this challenge, we introduce AutoAnalysis: an automated, self-updating meta-analysis of when humans and AI work better together, and when they work better alone. AutoAnalysis' most recent run synthesizes 1,955 effect sizes from 420 papers. Across this body of work, we found that, on average, human-AI combinations performed significantly better than humans alone, but they performed significantly worse than the better of humans or AI alone. These findings highlight the important distinction between improving human performance and achieving human-AI synergy and underscore the need to evaluate collaboration against both human and AI performance alone. As new studies emerge, AutoAnalysis will integrate their findings into the meta-analysis, providing cumulative evidence about when humans and AI perform better together and how human-AI performance varies across AI agents, tasks, and processes.

Comments:	Please visit this http URL to explore the data and view the most recent results
Subjects:	Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Digital Libraries (cs.DL)
Cite as:	arXiv:2610.42001 [cs.HC]
	(or arXiv:2610.42001v1 [cs.HC] for this version)
	https://doi.org/10.48550/arXiv.2610.42001 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Michelle Vaccaro [view email]
[v1] Fri, 30 Oct 2026 16:44:49 UTC (960 KB)