arXiv cs.AI· Kaiming Liu, Fuwen Luo, Ziyue Wang, Jinrui Ju, Yuxuan Liu, Xuanyu Lei, Yunghwei Lai, Peng Li, Yang Liu·· 3 天前AI 评分34
对齐的幻觉：检测协作对话中的隐藏分歧
Illusion of Alignment: Detecting Hidden Disagreement in Collaborative Dialogue
AI 导读
研究提出"对齐的幻觉（IoA）"概念，指协作对话表面达成一致、参与者却在目标或执行计划上仍有分歧，18 场真实会议研究证实其普遍存在。团队构建 IoA-Suite 数据集与评测协议，最佳模型仅达 49.5% F1，瓶颈在于对话未暴露的私有上下文；基于此训练的 IoA-Prober-8B 达到 51.8% F1，在 18 场真实会议中每场平均发现 2.89 处参与者确认未曾说出的隐藏分歧。
正文 · 原文
Abstract:Collaborative dialogue can end with apparent agreement while participants still differ on goals, assumptions, or execution plans, creating an \textbf{illusion of alignment (IoA)}. A real-user study across 18 meetings confirms that IoA arises routinely in human collaboration. Yet IoA poses a paradox: if participants were aware of such disagreements, they would already be explicit; if not, they cannot articulate them when asked, leaving IoA invisible to both participants and observers. In this work, we make IoA detectable by generating diagnostic multiple-choice questions whose divergent answers across participants provide direct behavioral evidence of hidden disagreement. We construct \textbf{IoA-Suite}, a dataset and evaluation protocol for detecting hidden disagreement, spanning five task types and six domains. We find that even the best model attains only 49.5\% F1, with the bottleneck traced to private context that the dialogue does not surface. We then train \textbf{IoA-Prober-8B} based on IoA-Suite, reaching 51.8\% F1 on IoA-Suite. Across the aforementioned 18 real meetings, it surfaces 2.89 hidden disagreements per meeting that participants confirm they had not voiced, transferring to live human dialogue. Further, in multi-agent collaboration, pairing IoA-Prober-8B with LLM agents improves downstream task performance on BigCodeBench-Hard and HiddenBench.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.08210 [cs.AI] |
| (or arXiv:2608.08210v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.08210 arXiv-issued DOI via DataCite |
Submission history
From: Kaiming Liu [view email]
[v1]
Sat, 8 Aug 2026 16:11:58 UTC (1,110 KB)
[v2]
Tue, 13 Oct 2026 07:15:19 UTC (2,639 KB)
来源：arXiv cs.AI · arxiv.org