跳到正文
arXiv cs.AI· Yufan Wang, Rui Yang, Yi Lin, Chunhua Weng, Yifan Peng·· 4 小时前AI 评分32

DiagTrace:评估 LLM 临床诊断决策的交互式基准

DiagTrace: An Interactive Benchmark for Clinical Decision Making

AI 导读

DiagTrace 是一个基于 PubMed Central 病例报告构建的交互式基准,用于评估 LLM 能否通过逐步信息检索重建真实诊断轨迹。基准含 8 个专科的 510 个病例,其中 225 个用于评测 4 个 LLM 在临床病程对齐、交互质量与效率上的表现,结果显示模型在信息覆盖、来源一致排序与对话轮次使用上存在权衡。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models (LLMs) have shown strong performance on medical question answering and diagnostic reasoning benchmarks. However, existing evaluations typically provide all patient information upfront and assess only the final diagnosis or rationale. This setup ignores the need to acquire diagnostic evidence incrementally from incomplete information in real-world diagnosis. To bridge this gap, we introduce DiagTrace, a source-grounded benchmark for evaluating whether LLMs can reconstruct the documented diagnostic trajectory through interactive information seeking. Each benchmark instance is built from a PubMed Central case report and includes an initial presentation, an event bank, a clinical course, and diagnostic references. DiagTrace contains 510 cases across eight specialties, with 225 used to evaluate four LLMs on clinical-course alignment, interaction quality, and efficiency. Results reveal trade-offs in coverage, source-consistent ordering, and turn use.
Comments: Accepted to Findings of EACL 2027
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22009 [cs.CL]
  (or arXiv:2611.22009v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.22009

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yufan Wang [view email]
[v1] Mon, 16 Nov 2026 23:42:28 UTC (166 KB)

来源:arXiv cs.AI · arxiv.org