arXiv cs.AI· Jason Armitage, Ioannis Tsochantaridis, Linda Mazzone, Chuqiao Yan, Srini Narayanan, Sarah Ebling·· 10 小时前AI 评分34
MAP:首个评估多模态 AI 无障碍出行规划能力的基准
MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places
AI 导读
研究者推出 MAP,首个评估多模态 AI 系统作为无障碍需求用户出行规划助手的基准。MAP 包含无障碍规划声明核验与视觉证据检索两项评估,检查系统能否判断地点无障碍信息是否有据、并选出对应地点的视觉证据。该基准采用自动评分加部分人工评分,并按计划时间刷新真值数据以应对信息随时间变化。
正文 · 原文
Abstract:We introduce MAP, the first benchmark to evaluate multimodal AI systems as assistants for users with accessibility requirements when planning visits to places in the real world. In our evaluation, systems are presented with requests to verify or recommend a point of interest meeting an accessibility requirement. MAP contains two novel assessments: Claim verification for accessibility planning assesses if information on places and stated accessibility features is supported and identifies places that satisfy requested accessibility features. Visual evidence retrieval for accessibility planning checks if a multimodal AI system can select visual evidence for the requested place and accessibility feature. Our methodology supports comparison of AI systems in a setting where place information and accessibility information can change over time by evaluating systems and refreshing ground truth data at scheduled times. The benchmark is based on automatic rating and human rating for a proportion of responses.
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7; I.2.10 |
| Cite as: | arXiv:2608.28384 [cs.AI] |
| (or arXiv:2608.28384v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.28384 arXiv-issued DOI via DataCite |
Submission history
From: Jason Armitage [view email]
[v1]
Fri, 28 Aug 2026 14:41:24 UTC (9,853 KB)
[v2]
Wed, 18 Nov 2026 12:53:35 UTC (7,379 KB)
来源:arXiv cs.AI · arxiv.org