arXiv cs.AI· Andre R Goncalves, Vincent Liu, Priyadip Ray·· 13 小时前AI 评分34
EHRAdapt：用语义先验将预训练语言模型适配到电子健康记录，提升罕见临床事件预测
EHRAdapt: Adapting Pretrained Language Models to Electronic Health Records with Semantic Priors for Rare Clinical Events
AI 导读
EHRAdapt 是一种适配器，可将电子健康记录的（时间、模态、编码）元组直接映射进冻结语言模型的嵌入空间，仅训练 0.1–0.6% 的参数。研究在约 400 万患者记录上对 OLMo2 1B、Llama3.2 1B 和 OLMo2 7B 三个冻结骨干做继续预训练，完整适配器在每个骨干的留出下一事件预测上均优于所有消融版本。
正文 · 原文
Abstract:Electronic health records (EHRs) encode clinical histories as (time, modality, code) tuples, whereas pretrained language models expect text tokens. Serializing tuples as text inflates sequence length and repeatedly encodes the same structure. We introduce EHRAdapt, an adapter that maps tuples directly into a frozen language model's embedding space. Modality and time gaps enter through a learned embedding and attention biases, and event codes receive dedicated vectors. Learning them is challenging because clinical vocabularies are long-tailed, leaving rare events with few observations. EHRAdapt therefore represents each event vector as the sum of a semantic prior and an evidence residual. The prior is a frozen, ontology-trained biomedical embedding of the event's clinical description, mapped into the input space by a shared learned projection, so it supplies clinical meaning even when observations are scarce. The residual is a learned low-rank, event-specific correction that refines it as evidence accumulates. We perform continued pretraining on records from about 4 million patients with three frozen LLM backbones, OLMo2 1B, Llama3.2 1B, and OLMo2 7B, training only the adapter (0.1--0.6% of all parameters). The full adapter outperforms all ablations in held-out next-event prediction on every backbone. Removing the semantic pathway hurts rare events most, over ten times more than the most frequent ones, whereas removing the residual hurts overall prediction but improves it for the rarest events. On reportable infectious-disease and syndromic classification, EHRAdapt outperforms text-based LLM and count-based baselines and the same LLMs adapted to the same records through next-event prediction, and both pathways improve rare-disease discrimination. The two pathways therefore play complementary roles, visible only when results are broken down by event frequency.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.34007 [cs.AI] |
| (or arXiv:2609.34007v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.34007 arXiv-issued DOI via DataCite |
Submission history
From: Andre Goncalves [view email]
[v1]
Sun, 27 Sep 2026 23:07:08 UTC (297 KB)
[v2]
Tue, 20 Oct 2026 19:02:37 UTC (300 KB)
来源：arXiv cs.AI · arxiv.org