跳到正文
arXiv cs.CL· Antonio Puertas Gallardo, Mario Ceresa·· 10 小时前AI 评分31

No-Harm-Jury:面向医疗 AI 回答的陪审团式多维度危害评估框架

No-Harm-Jury A Jury-Based Multi-Dimensional Harm Evaluation Framework for Medical AI Responses

AI 导读

No-Harm-Jury 是一个陪审团式自动评估框架,用五个不同语言模型作为独立陪审员,从信息、社会、心理、自主性、经济、隐私和认知七个危害维度为 AI 医疗回答打分。

正文 · 原文

View PDF HTML (experimental)

Abstract:We present No-Harm-Jury, a jury-based automated evaluation framework that scores AI-generated medical responses across seven conceptually distinct harm dimensions using a panel of five diverse language models as independent jurors. Responses generated by Ministral-14B on 3,000 questions drawn from PubMedQA, MedQA, and MedMCQA are scored by five jurors across dimensions of informational, social, psychological, autonomy, economic, privacy, and epistemic harm; the 151 empty responses (98 MedQA, 53 MedM- CQA) are excluded, leaving 2,849 valid samples. The median aggregation robustly suppresses individual juror biases while maintaining coverage through complementary specialisation. No existing system combines all five prop- erties of No-Harm-Jury: automated scalable evaluation, five diverse LLM jurors, robust me- dian aggregation (with a reliability-weighted extension), seven conceptually distinct harm dimensions, and application to medical QA benchmarks.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24972 [cs.CL]
  (or arXiv:2611.24972v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.24972

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Antonio Puertas Gallardo [view email]
[v1] Mon, 16 Nov 2026 14:08:54 UTC (180 KB)

来源:arXiv cs.CL · arxiv.org