arXiv cs.AI· Keyty Loza, Shayan Monadjemi·· 11 小时前AI 评分26
视觉数据面前，AI 智能体如何做决策？
How Do Artificial Agents Make Decisions in Light of Visual Data?
AI 导读
一项研究评估了由 VLM 驱动的 AI 智能体在视觉数据下感知不确定性与自主决策的表现。在测试模型中，GPT-4o 对饼图和柱状图中不确定性的感知准确率最高；当不确定性以圆形图、图标阵列或文本呈现时，智能体的决策与理性决策理论吻合度最佳。
正文 · 原文
Abstract:Vision-language models (VLMs) have demonstrated significant advances in reasoning with visual data, which is particularly important given that much of human knowledge is encoded visually in scientific articles and other sources. However, a gap remains in understanding how artificial agents, powered by VLMs, perceive and navigate decision-making under uncertainty in the presence of visual data. This study explores this gap by evaluating and analyzing autonomous decision-making processes. Building upon prior work in visualization literacy and decision theory, we conducted two complementary studies: one examining how agents perceive and quantify uncertainty encoded in visualizations, and another investigating how artificial agents make decisions based on visual data in the form of images. We analyzed perception accuracy and decision performance across models and visualization types. Our results show that GPT-4o demonstrates the highest accuracy in perceiving uncertainty in pie and bar charts among the tested models. For decision-making, autonomous agents showed the best alignment with rational decision theory when uncertainty was represented through circle charts, icon arrays, or text.
| Comments: | Presented at the Workshop on GenAI, Agents, and the Future of VIS, held in conjunction with IEEE VIS, 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2611.05671 [cs.AI] |
| (or arXiv:2611.05671v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05671 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shayan Monadjemi [view email]
[v1]
Tue, 3 Nov 2026 21:58:10 UTC (609 KB)
来源：arXiv cs.AI · arxiv.org