跳到正文
arXiv cs.CL· Feras AlMannaa, Talia Tseriotou, Jenny Chim, Maria Liakata·· 2 天前AI 评分38

研究 LLM 在临床问答中的长上下文理解能力

Investigating LLM Capabilities on Long Context Comprehension for Clinical Question Answering

AI 导读

一项覆盖多数据集、多模型与多指标的研究系统考察了 LLM 对纵向患者电子健康记录(EHR)的长上下文理解能力,涵盖抽取式、多项选择与开放式生成问答,并对比不同上下文构造与 RAG 设置。

正文 · 原文

View PDF HTML (experimental)

Abstract:Despite the impressive reported performance of Large Language Models (LLMs) on medical Question-Answering (QA), evaluation still overrelies on multiple-choice and extractive settings primarily assessing textbook knowledge, with limited work on open-ended QA over patient records. We present a comprehensive study investigating LLM long-context (LC) comprehension on longitudinal, patient-centric Electronic Health Records (EHRs) across multiple datasets, models and metrics, covering extractive, multiple-choice, and open-ended generative QA under different context formulation and Retrieval Augmented Generation (RAG) settings. We show that medically fine-tuned models offer no measurable benefit over base counterparts, smaller models exhibit stronger memorization, and longitudinal supplementary notes aid generative QA. Reasoning models gain on complex multi-note QA under LLM Correctness but not stricter reference-aligned metrics, while hybrid RAG with reranking outperforms Full-Context (FC) prompting. A question-category analysis and multi-faceted metric evaluation further reveal where RAG and FC excel and struggle, with RAG falling behind in longitudinal reasoning, thus uncovering systematic failures pointing to the need for reasoning-appropriate metrics.
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2510.18691 [cs.CL]
  (or arXiv:2510.18691v3 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2510.18691

arXiv-issued DOI via DataCite

Submission history

From: Feras AlMannaa [view email]
[v1] Tue, 21 Oct 2025 14:50:24 UTC (2,482 KB)
[v2] Wed, 14 Jan 2026 21:40:27 UTC (2,585 KB)
[v3] Thu, 12 Nov 2026 22:58:31 UTC (2,747 KB)

来源:arXiv cs.CL · arxiv.org