arXiv cs.CL· Jen-Tse Huang, Dasen Dai, Jen-Yuan Huang, Youliang Yuan, Xiaoyuan Liu, Wenxuan Wang, Wenxiang Jiao, Pinjia He, Zhaopeng Tu, Haodong Duan·· 13 小时前AI 评分58
人类认知基准 VisFacor 揭示多模态大模型的基础视觉短板
Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs
AI 导读
研究团队提出 VisFacor 基准，将认知心理学评估 FRCT 中 20 项视觉中心子测验数字化，并设计算法自动生成难度可控的测试用例。在 39 个前沿多模态大模型上评测，最高分仅 55.9%，模型在心理旋转、空间关系推理和图形背景区分等任务上普遍失败，与模型规模和提示策略无关。该基准内部一致性 Cronbach's alpha 为 0.94，论文已被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Humans develop perception through a bottom-up hierarchy: from basic primitives and Gestalt principles to high-level semantics. Current Multimodal Large Language Models (MLLMs) are trained directly on complex downstream tasks. Does it guarantee their performance on these foundational visual capabilities? To systematically investigate this gap, we introduce VisFacor, a benchmark that digitizes 20 vision-centric subtests from FRCT, a well-established cognitive psychology assessment spanning four domains of human visual cognition. Furthermore, we design algorithms to automatically construct and validate unlimited test cases with controllable difficulty. Using VisFacor, we evaluate 39 frontier MLLMs, including both proprietary (e.g., GPT, Gemini) and open-source (e.g., LLaMA, Qwen) models. The best model achieves a score of only 55.9%. Analysis reveals good internal consistency (Cronbach's alpha = 0.94) and construct validity (compared to existing vision benchmarks). Models consistently fail on tasks such as mental rotation, spatial relation inference, and figure-ground discrimination, regardless of model size or prompting strategy. These findings suggest that performance improvements on existing general benchmarks might represent castles in the air instead of a genuine mastery of human-like visual cognition.
| Comments: | Accepted to NeurIPS 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2502.16435 [cs.CV] |
| (or arXiv:2502.16435v5 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2502.16435 arXiv-issued DOI via DataCite |
Submission history
From: Jen-Tse Huang [view email]
[v1]
Sun, 23 Feb 2025 04:21:32 UTC (872 KB)
[v2]
Thu, 7 Aug 2025 15:39:27 UTC (2,102 KB)
[v3]
Tue, 27 Jan 2026 15:14:06 UTC (3,296 KB)
[v4]
Sun, 3 May 2026 19:46:16 UTC (3,692 KB)
[v5]
Wed, 21 Oct 2026 01:48:56 UTC (3,757 KB)
来源：arXiv cs.CL · arxiv.org