arXiv cs.AI· Ashvin Gupta, Brendan Delaney, Alessandra Russo·· 13 小时前AI 评分22
JEPA 用于纵向 EHR 序列表示是否有效？基于 MIMIC-IV 的实证研究
Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?
AI 导读
研究在 MIMIC-IV 上检验 Joint-Embedding Predictive Architectures（JEPA）能否用于纵向电子健康记录（EHR）序列表示，三种 JEPA 形式均训练稳定、无表示坍缩，但下游表现一致低于标准 MLM。目标尺寸而非目标异质性会增大预测误差，缩小目标尺寸也未恢复下游性能，说明仅靠稳定的潜空间预测不足以学到有竞争力的 EHR 表示。
正文 · 原文
Abstract:Masked language modelling (MLM) is the dominant pretraining objective for transformer encoders applied to longitudinal electronic health records (EHRs). However, reconstructing individual events may overemphasise noisy or weakly informative observations and does not explicitly encourage learning higher-level patient states. Joint-Embedding Predictive Architectures (JEPAs) instead learn by predicting representations in latent space and have shown strong performance in a variety of domains. In this paper, we investigate to what extent JEPA can be adapted to EHR sequence representation. Our empirical investigation using MIMIC-IV shows that although all three of our JEPA formulations train stably without representational collapse, they consistently underperform standard MLM. We find that target size, not target heterogeneity, increases prediction error, yet reducing the target size does not recover downstream performance. This suggests that stable latent prediction alone is insufficient for competitive EHR representation learning and highlights the challenge of defining clinically meaningful latent targets for EHR.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.29528 [cs.LG] |
| (or arXiv:2610.29528v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29528 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ashvin Gupta [view email]
[v1]
Wed, 21 Oct 2026 10:18:40 UTC (989 KB)
来源：arXiv cs.AI · arxiv.org