arXiv cs.CL· Hamza Shafiq, Jialu Tang, Ting Dang, Jun Hu, Aaqib Saeed·· 6 天前AI 评分22
PRISM：融合异构证据的检索增强多模态语言模型用于临床预后
Retrieval-Augmented Multimodal Language Model with Heterogeneous Evidence Integration for Clinical Prognosis
AI 导读
研究者提出 PRISM（Population-Retrieved Inference for Structured Multimodal Prognosis），一种融合逐特征数值 token 化、纵向 ECG 近因加权聚合与双模态人群记忆库的多模态语言模型，检索在表示空间进行，不将单条记录写入提示词。
正文 · 原文
Abstract:Clinical prognosis requires integrating structured measurements, physiological waveforms, longitudinal history, and population context. We introduce PRISM (Population-Retrieved Inference for Structured Multimodal Prognosis), a multimodal language model that combines per-feature numerical tokenization, recency-weighted aggregation of longitudinal ECGs, and a dual-modality population memory bank. Retrieval operates in representation space: the model receives mean-pooled embeddings of similar training-cohort encounters, while no individual retrieved record enters the prompt. On the 1,443-task MDS-ED benchmark, PRISM achieves AUROCs of 86.2% for diagnosis, 93.1% for deterioration, 91.8% for ICU admission, and 92.8% for mortality. These results use a 96-hour pre-arrival window unavailable to reported baselines. With benchmark-window inputs only, PRISM reaches 0.8895 AUROC, compared with 0.8937 for UniPACT and 0.8889 for MDS-ED. A matched-input compact fusion model using the same five evidence streams and frozen encoders reaches 0.7652 AUROC, versus 0.9092 for PRISM. Replacing numerical EHR encoding with text serialization reduces AUROC by 3.66 points; removing pre-arrival history reduces it by 1.97 points; and zeroing retrieval reduces it by 0.99 points. Genuine retrieval outperforms random-neighbor retrieval by 0.97 macro-average points, although a paired task-level sign test does not establish a uniform advantage (p=0.067). On the external MC-MED cohort, supervised fine-tuning reaches 0.8611 AUROC, compared with 0.8227 for locally trained gradient boosting and 0.7841 for source-only PRISM. These results support structure-aware multimodal integration and supervised external adaptation.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13510 [cs.AI] |
| (or arXiv:2610.13510v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13510 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hamza Shafiq [view email]
[v1]
Fri, 9 Oct 2026 12:35:22 UTC (1,632 KB)
来源：arXiv cs.CL · arxiv.org