arXiv cs.CL· Antonio Puertas Gallardo, Mario Ceresa·· 10 小时前AI 评分31
No-Harm-Jury:面向医疗 AI 回答的陪审团式多维度危害评估框架
No-Harm-Jury A Jury-Based Multi-Dimensional Harm Evaluation Framework for Medical AI Responses
AI 导读
No-Harm-Jury 是一个陪审团式自动评估框架,用五个不同语言模型作为独立陪审员,从信息、社会、心理、自主性、经济、隐私和认知七个危害维度为 AI 医疗回答打分。
正文 · 原文
Abstract:We present No-Harm-Jury, a jury-based automated evaluation framework that scores AI-generated medical responses across seven conceptually distinct harm dimensions using a panel of five diverse language models as independent jurors. Responses generated by Ministral-14B on 3,000 questions drawn from PubMedQA, MedQA, and MedMCQA are scored by five jurors across dimensions of informational, social, psychological, autonomy, economic, privacy, and epistemic harm; the 151 empty responses (98 MedQA, 53 MedM- CQA) are excluded, leaving 2,849 valid samples. The median aggregation robustly suppresses individual juror biases while maintaining coverage through complementary specialisation. No existing system combines all five prop- erties of No-Harm-Jury: automated scalable evaluation, five diverse LLM jurors, robust me- dian aggregation (with a reliability-weighted extension), seven conceptually distinct harm dimensions, and application to medical QA benchmarks.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24972 [cs.CL] |
| (or arXiv:2611.24972v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24972 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Antonio Puertas Gallardo [view email]
[v1]
Mon, 16 Nov 2026 14:08:54 UTC (180 KB)
来源:arXiv cs.CL · arxiv.org