跳到正文
arXiv cs.CY· Ruio Heerschap, Henry Foote, Mahmoud Alwakeel, Zhouhan Qian, Jiaqi Wang, Elisabeth Bond, Yang Yang, Jonathan Chong Kai Liew, Matthew A. Bergens, Monique Arnold, Haoyuan Wang, Rui Yang, Kunyu Yu, Nikita R. Paripati, Diana Pimenta, Kyle Naimoli, An-Kwok Ian Wong, Nicoleta Economou-Zavlanos, Jo\~ao Ricardo Vissoci, Nan Liu, Anand Chowdhury, Chuan Hong·· 9 小时前AI 评分31

SAFE-Triage:面向患者的分诊 AI 工具多维评估框架

Beyond Acuity Accuracy: A Multidimensional Framework for Evaluating Patient-Facing Emergency Triage AI Tools

AI 导读

研究团队提出 SAFE-Triage 框架,整合人工审核、自动化指标、LLM-as-judge 评估与模拟测试,从临床性能、患者交互和面向临床医生的摘要质量三个维度评估面向患者的分诊 AI 工具。

正文 · 原文

Authors:Ruio Heerschap, Henry Foote, Mahmoud Alwakeel, Zhouhan Qian, Jiaqi Wang, Elisabeth Bond, Yang Yang, Jonathan Chong Kai Liew, Matthew A. Bergens, Monique Arnold, Haoyuan Wang, Rui Yang, Kunyu Yu, Nikita R. Paripati, Diana Pimenta, Kyle Naimoli, An-Kwok Ian Wong, Nicoleta Economou-Zavlanos, João Ricardo Vissoci, Nan Liu, Anand Chowdhury, Chuan Hong

View PDF

Abstract:Patient-facing AI triage tools may support care-seeking and emergency department prioritization, but require evaluation beyond acuity accuracy alone. We developed SAFE-Triage, a framework integrating human review, automated metrics, LLM-as-judge evaluation, and simulation testing across clinical performance, patient interaction, and clinician-facing summary quality. We applied SAFE-Triage to 40 MIMIC-IV-derived emergency vignettes and a 10,030-case acuity analysis comparing prompted LLM triage (PLT) with rule-augmented prompted LLM triage (rPLT). PLT achieved higher emergency severity index (ESI) accuracy (57.5% vs 42.8%, $P < 0.001$), lower undertriage (13.1% vs 29.8%, $P < 0.001$), and higher clinician ratings across all domains. rPLT produced shorter and more stable outputs with higher automated readability scores. SAFE-Triage identified trade-offs between acuity accuracy, undertriage risk, communication quality, readability, and summary quality, supporting pre-deployment refinement and ongoing monitoring by showing where triage tools are efficient, need improvement, or need clinician oversight.
Comments: 82 pages, including 34 pages of supplementary material. 5 figures (4 main figures and 1 supplementary figure)
Subjects: Computation and Language (cs.CL); Computers and Society (cs.CY)
Cite as: arXiv:2611.22045 [cs.CL]
  (or arXiv:2611.22045v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.22045

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zhouhan Qian [view email]
[v1] Tue, 17 Nov 2026 01:11:07 UTC (2,290 KB)

来源:arXiv cs.CY · arxiv.org