arXiv cs.CY· Kiant\'e Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C. Miceli, Ian Krajbich·· 11 小时前AI 评分47
CogArena：在交互式行为实验中评测多模态智能体
CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments
AI 导读
CogArena 是一个包含十种社会与行为科学实验范式的基准，智能体通过标准网页浏览器、使用与在线被试相同的视觉刺激和响应界面完成实验。评测采用三级评分，分别考察任务是否完成、完成准确度以及数据是否呈现该范式在人类身上稳定出现的行为特征。对四个多模态模型各任务运行十次后发现，完成实验与复现行为特征并不等同：许多会话未能完成，完成的会话中部分特征可靠复现、部分完全无法复现。
正文 · 原文
Abstract:Claims about the cognitive capabilities of large language models are typically tested by translating behavioral paradigms into text prompts, bypassing the visual interfaces and time pressure through which human participants actually engage with experiments. At the same time, AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination. We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences. Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use. A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people. Across four multimodal models, each run ten times per task, we find that finishing an experiment and reproducing its signatures are different things. Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all. CogArena provides the tasks, scoring code, and a public leaderboard for new submissions.
| Comments: | 28 pages, 4 figures, 11 tables. Accepted at NeurIPS 2026 (Evaluations & Datasets Track). Code: this https URL. Leaderboard: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC); Neurons and Cognition (q-bio.NC) |
| Cite as: | arXiv:2610.36484 [cs.AI] |
| (or arXiv:2610.36484v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36484 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kiante Fernandez [view email]
[v1]
Tue, 27 Oct 2026 01:42:35 UTC (262 KB)
来源：arXiv cs.CY · arxiv.org