HealthAgentBench：面向前沿 AI 智能体的真实医疗智能体环境统一基准套件
HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents
HealthAgentBench 发布，包含覆盖 9 个类别、各配独立环境的 75 项医疗智能体任务，涵盖诊断、标本评估、研究与数据管理流程。
Authors:Qianchu Liu, Sheng Zhang, Guanghui Qin, Jeya Maria Jose Valanarasu, Xiaoke Huang, Peniel Argaw, Maximilian Rokuss, Mingyu Lu, Timothy Ossowski, Juan Manuel Zambrano Chaves, Cliff Wong, Naisargi Patel, Yashna Hasija, Mu Wei, Wen-wai Yim, Qin Liu, Zilin Jing, Jason Entenmann, Naoto Usuyama, Tristan Naumann, Hoifung Poon
Abstract:As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous evaluation is essential for measuring progress toward real-world healthcare applications. However, existing healthcare benchmarks are often static or narrow in clinical scope, motivating broader evaluation of agents acting on raw patient data. We introduce HealthAgentBench, a suite of 75 agentic healthcare tasks across 9 categories each with its unique environment. The benchmark suite spans diagnosis, specimen assessment, research, and data management workflows and a broad range of modalities. Each task is designed to replicate an end-to-end clinical workflow: given task instructions and supporting resources, an agent must explore raw healthcare data, operate within a complex environment, and execute multi-step solutions that go beyond naive prompting, such as querying large clinical databases or interpreting gigapixel pathology images. All tasks are scored using binary success criteria derived from reference labels, executable rules, or computational baselines. We evaluate six agents (three GPT models through Codex and three Claude models through Claude Code) with one scored attempt per task (450 evaluations). Overall success ranges from 32.0% to 57.3%; Claude Code Opus-5.5 leads with a 57.3% success rate, followed by GPT-6-astra and Sonnet-5.5 at 50.7% each. Performance varies substantially by category with specialized imaging modalities such as CT and MRI being especially difficult. HealthAgentBench provides a challenging and realistic benchmark with substantial room for future progress. We release our benchmark athttps://github.com/microsoft/HealthAgentBench.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2606.31179 [cs.AI] |
| (or arXiv:2606.31179v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.31179 arXiv-issued DOI via DataCite |
Submission history
From: Qianchu Liu [view email]
[v1]
Tue, 30 Jun 2026 06:09:01 UTC (2,438 KB)
[v2]
Sun, 25 Oct 2026 02:14:40 UTC (1,059 KB)
来源：arXiv cs.CL · arxiv.org