arXiv cs.AI· Michaela Benk, Kacper Sokol·· 3 天前AI 评分18
信任阻碍了双向人机对齐
Trust Hinders Bidirectional Human-AI Alignment
AI 导读
研究指出，将信任校准作为 AI 辅助决策的核心框架过于局限：信任 AI 只是管理不确定性的策略之一，且当 AI 面对目标与状态仅部分可知或不可知的"大世界"决策问题时，信任校准常被误用。在此类场景中，人机对齐需处理更高阶的决策过程，作者据此提出面向过程的人机耦合系统设计思路。
正文 · 原文
Abstract:Trust calibration remains the dominant framing of AI-assisted decision making: the goal is to close the gap between perceived and actual trustworthiness of AI systems so that users exhibit appropriate reliance on them. We argue that this approach, while useful for decision problems that are well-defined and clearly scoped, overly restricts what it means to make a decision with AI support. To overcome this limitation, we develop three dimensions of AI-assisted decision making. First, trusting AI is only one of several possible strategies for managing uncertainty. Second, trust calibration is often misapplied when AI models "large-world" decision problems, i.e., ill-defined situations where goals and states are only partially known or unknowable. Third, in such settings human-AI alignment requires addressing higher-order decision processes. We conclude by reflecting on the implications of our observations on the design of process-oriented human-AI coupled systems that jointly identify and negotiate uncertainty throughout the decision-making task.
| Comments: | 40th Conference on Neural Information Processing Systems (NeurIPS 2026). Workshop: Dynamic Alignment in Human-AI Coupled System (BiAlign Workshop) |
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2610.15136 [cs.AI] |
| (or arXiv:2610.15136v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15136 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Michaela Benk [view email]
[v1]
Sat, 10 Oct 2026 19:26:37 UTC (43 KB)
来源：arXiv cs.AI · arxiv.org