跳到正文
arXiv cs.AI· Priyanka Sinha, Sugam Pandey·· 10 小时前AI 评分40

元认知校准决定人机协作中人类审阅者的价值:一项基于 AI 辅助会计系统的特质理论研究

Knowing What You Know: A Trait Theory of Reviewer Value in Human-AI Cognitive Systems

AI 导读

研究提出“审阅者价值特质理论”,认为在人类与 AI 模型协作决策时,决定人类审阅者价值的核心特质是元认知校准,即行为置信度与实际判断正确性的一致程度。基于 AI 辅助会计系统六个月内 541621 条审阅事件、483 名审阅者的数据,纯行为专家画像会把过度自信群体(5.2% 纠错存活率)排在真正专家(99.2%)之上,且审阅者质量在不同任职时长上基本持平。

正文 · 原文

View PDF HTML (experimental)

Abstract:When a capable model and a human reviewer cooperate on the same decision, which human still adds value? We argue this is a question about a single cognitive trait, metacognitive calibration: the alignment between a reviewer's behavioral confidence and the actual correctness of their judgments. We develop a trait theory of reviewer value to make it precise. The theory makes five claims: value is governed by calibration rather than by activity, speed, agreement, or tenure; calibration leaves observable traces in behavioral telemetry; those traces are non-diagnostic on their own, because the fast, low-revision signature of a calibrated expert is indistinguishable from that of a confident novice until it is gated by outcomes; human calibration and a model's self-knowledge are the same construct, so the joint system should defer to whichever agent is better calibrated on each item; and individual calibration is causally identifiable only above the level of the single judgment. We ground each claim in a six-month production window of an AI-assisted accounting system (541621 reviewer events, 483 reviewers), and find that a purely behavioral expertise profile inverts the ground truth, scoring an overconfident cohort (5.2 percent correction survival) higher than genuine experts (99.2 percent); that reviewer quality is flat across tenure, so the trait is not learned on the job; and that the same calibration logic that separates good from bad reviewers, applied to the model, concentrates 92.5 percent of human effort on genuine errors. We close with the boundary the theory must respect: under majority-vote aggregation, individual causal value is unidentifiable.
Comments: Accepted at the Thirteenth Annual Conference on Advances in Cognitive Systems (ACS), Edinburgh, United Kingdom, October 6-9, 2026. Awarded one of 2026 Ada Lovelace Awardees for Excellent Paper Submissions to ACS
Subjects: Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Cite as: arXiv:2611.24932 [cs.AI]
  (or arXiv:2611.24932v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.24932

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Priyanka Sinha [view email]
[v1] Wed, 18 Nov 2026 18:55:07 UTC (38 KB)

来源:arXiv cs.AI · arxiv.org