arXiv cs.CL· Caixin Kang, Yifei Huang, Liangyang Ouyang, Mingfang Zhang, Ruicong Liu, Yoichi Sato·· 2 天前AI 评分28
MLLMs 能读懂房间吗？评估多方社交互动中欺骗行为的多模态基准
Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions
AI 导读
研究者提出 MIDA——一个包含同步视频与文本、每条陈述均有可验证真值标签的多模态数据集，用于评估 MLLMs 在多方社交互动中的欺骗识别能力。对 12 个开源与闭源 MLLMs 的基准测试显示，即便 GPT-4o 也难以可靠区分真实与欺骗性陈述。
正文 · 原文
Abstract:Despite their advanced reasoning capabilities, state-of-the-art Multimodal Large Language Models (MLLMs) still lack a core component of human intelligence: the ability to "read the room" and assess deception in complex social interactions. To rigorously evaluate this limitation, we introduce Multimodal Interactive Deception Assessment (MIDA), a new task and multimodal dataset comprising synchronized video and text with verifiable ground-truth labels for every statement. We benchmark 12 state-of-the-art open- and closed-source MLLMs and identify a substantial performance gap: even powerful models such as GPT-4o struggle to reliably distinguish truthful statements from deceptive ones. Our analysis shows that current models have difficulty grounding language in multimodal social cues and modeling what others know, believe, or intend. To address these limitations, we propose a Social Chain-of-Thought (SoCoT) reasoning pipeline and a Dynamic Social Epistemic Memory (DSEM) module. Experimental results show that structured epistemic memory consistently improves deception assessment, while multimodal social-cue reasoning remains a challenging and important direction for future research.
| Comments: | Accepted to EMNLP 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2511.16221 [cs.CV] |
| (or arXiv:2511.16221v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2511.16221 arXiv-issued DOI via DataCite |
Submission history
From: Caixin Kang [view email]
[v1]
Thu, 20 Nov 2025 10:44:21 UTC (4,361 KB)
[v2]
Thu, 15 Oct 2026 14:15:08 UTC (7,065 KB)
来源：arXiv cs.CL · arxiv.org