arXiv cs.AI· Seohyun Lee, Jongha Kim, Dohwan Ko, Ji Soo Lee, Seoung Choi, Jayun Ku, Hyunwoo J. Kim·· 12 小时前AI 评分38
MMEnterprise-Bench：评估企业智能体在动态多模态记录上的表现
MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records
AI 导读
研究者推出 MMEnterprise-Bench，一个包含 1,613 道题的企业智能体基准，题目基于模拟协作项目中不断演变的多模态记录，覆盖证据推理、冲突消解和不可回答三类。对 10 个闭源与开源多模态 LLM 的评测显示，即使最强模型在跨来源或时间冲突、以及跨多张图片的证据上也表现吃力。提高检索预算或换用更大的检索器能提升证据召回率，却未稳定转化为更高的答题准确率。
正文 · 原文
Abstract:Enterprise agents answer questions over workplace records that evolve through team collaboration. As information is shared, corrected, and revised across documents, chats, and figures, multiple versions of the same information accumulate in the workspace. Reliable question answering therefore requires agents to reason over sufficient evidence, resolve cross-source and temporal conflicts, and recognize when the available evidence is insufficient to establish an answer. To systematically evaluate these capabilities, we introduce MMEnterprise-Bench, a benchmark comprising 1,613 questions grounded in evolving multimodal records from simulated collaboration projects, covering three categories: Evidence Reasoning, Conflict Resolution, and Unanswerability. We evaluate 10 proprietary and open-source multimodal LLMs and find that even the strongest models struggle when evidence contains cross-source or temporal conflicts or spans multiple images. Moreover, increasing the retrieval budget or employing a larger retriever improves evidence recall but does not consistently translate into higher answer accuracy. These findings highlight the persistent challenges of reasoning over evolving workplace records and establish MMEnterprise-Bench as a testbed for developing more reliable enterprise agents. Code and data are available at this https URL.
| Comments: | 35 pages, 16 figures, including appendices. Under review |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.27911 [cs.AI] |
| (or arXiv:2610.27911v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27911 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Seohyun Lee [view email]
[v1]
Tue, 20 Oct 2026 09:25:35 UTC (5,777 KB)
来源：arXiv cs.AI · arxiv.org