arXiv cs.AI· Elaine Lau, Antoine Pangas, Alexandre Finot, Collin Schweiker, Amir Bouzidi, Hui Wen Goh, Andrew Peter Martin, Jonas Mueller·· 4 小时前AI 评分58
ATLAS-Finance:在真实银行环境中评测 AI 智能体的基准
ATLAS-Finance: Benchmarking AI Agents in Realistic Banking Environments
AI 导读
ATLAS-Finance 是一个包含 100 个端到端投资银行任务、覆盖 13 家模拟公司的基准,由资深银行家基于真实交易工作设计,单个任务需 15-30 小时完成。智能体需在邮件、聊天、日历、笔记和文件中检索信息、调和冲突与更新,再构建复杂财务模型,交付物由专家撰写的评分标准评估(中位数 124.5 条)。当前前沿模型中表现最好的 Claude Opus 和 Fable 通过率不足 11%。
正文 · 原文
Abstract:Professional work involves under-specified assignments, where relevant information is spread across workplace applications and may conflict or change over time. To evaluate how well frontier agents can perform economically valuable work in such settings, we introduce ATLAS-Finance, a benchmark of 100 end-to-end investment-banking tasks across 13 simulated firms. As in real banking workflows, agents must search email, chat, calendars, notes, and files to identify requirements, gather information, reconcile conflicts/updates, and then build complex financial models. Tasks were developed by experienced bankers based on their real deal work and take them 15-30 hours to complete. Agent deliverables are evaluated with expert-authored rubrics (median: 124.5 criteria) covering financial rationale and results, spreadsheet construction, completeness, and formatting. Despite completing substantial portions of the requested work, current agents rarely produce correct deliverables. Across today's frontier models, the best ones (Claude Opus and Fable) pass fewer than 11% of tasks.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22085 [cs.AI] |
| (or arXiv:2611.22085v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22085 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Elaine Lau [view email]
[v1]
Tue, 17 Nov 2026 02:12:40 UTC (336 KB)
来源:arXiv cs.AI · arxiv.org