arXiv cs.CL· Feras AlMannaa, Talia Tseriotou, Jenny Chim, Maria Liakata·· 2 天前AI 评分38
研究 LLM 在临床问答中的长上下文理解能力
Investigating LLM Capabilities on Long Context Comprehension for Clinical Question Answering
AI 导读
一项覆盖多数据集、多模型与多指标的研究系统考察了 LLM 对纵向患者电子健康记录(EHR)的长上下文理解能力,涵盖抽取式、多项选择与开放式生成问答,并对比不同上下文构造与 RAG 设置。
正文 · 原文
Abstract:Despite the impressive reported performance of Large Language Models (LLMs) on medical Question-Answering (QA), evaluation still overrelies on multiple-choice and extractive settings primarily assessing textbook knowledge, with limited work on open-ended QA over patient records. We present a comprehensive study investigating LLM long-context (LC) comprehension on longitudinal, patient-centric Electronic Health Records (EHRs) across multiple datasets, models and metrics, covering extractive, multiple-choice, and open-ended generative QA under different context formulation and Retrieval Augmented Generation (RAG) settings. We show that medically fine-tuned models offer no measurable benefit over base counterparts, smaller models exhibit stronger memorization, and longitudinal supplementary notes aid generative QA. Reasoning models gain on complex multi-note QA under LLM Correctness but not stricter reference-aligned metrics, while hybrid RAG with reranking outperforms Full-Context (FC) prompting. A question-category analysis and multi-faceted metric evaluation further reveal where RAG and FC excel and struggle, with RAG falling behind in longitudinal reasoning, thus uncovering systematic failures pointing to the need for reasoning-appropriate metrics.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2510.18691 [cs.CL] |
| (or arXiv:2510.18691v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2510.18691 arXiv-issued DOI via DataCite |
Submission history
From: Feras AlMannaa [view email]
[v1]
Tue, 21 Oct 2025 14:50:24 UTC (2,482 KB)
[v2]
Wed, 14 Jan 2026 21:40:27 UTC (2,585 KB)
[v3]
Thu, 12 Nov 2026 22:58:31 UTC (2,747 KB)
来源:arXiv cs.CL · arxiv.org