JEPA-DNA：用联合嵌入预测架构为基因组基础模型注入功能语义
JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures
JEPA-DNA 是一个模型无关的持续训练框架，将联合嵌入预测架构（JEPA）与生成式目标结合，通过监督潜空间中的全局序列嵌入，让模型预测被掩码基因组片段的功能表征，把学习信号从 token 还原转向语义对齐。在 17 项基因组基准任务上，无论底层 GFM 架构或生成目标如何，线性探测与零样本性能均获一致提升，并在线性探测协议下刷新 GFM 的 SOTA。代码见补充材料。
Authors:Ariel Larey, Elay Dahan, Amit Bleiweiss, Raizy Kellerman, Guy Leib, Omri Nayshool, Dan Ofer, Tal Zinger, Dan Dominissini, Gideon Rechavi, Nicole Bussola, Simon Lee, Shane O'Connell, Dung Hoang, Marissa Wirth, Alexander W. Charney, Nati Daniel, Yoli Shavit
Abstract:Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local syntax, these generative paradigms prioritize token-level reconstruction over high-level functional context. We introduce JEPA-DNA, a model-agnostic continual training framework that integrates a Joint-Embedding Predictive Architecture (JEPA) with traditional generative objectives. By supervising global sequence embeddings in a latent space, JEPA-DNA forces models to predict the functional representations of masked genomic segments, shifting the learning signal from token recovery to semantic alignment. We evaluate JEPA-DNA on 17 diverse genomic benchmark tasks, demonstrating consistent gains in linear probing and zero-shot performance regardless of the underlying GFM architecture or generative objective, and establishing a new state-of-the-art for GFMs under the linear-probing protocol. Through extensive ablation studies, we further characterize the synergistic interplay between generative and latent objectives. Our code is provided in the supplementary material.
| Subjects: | Artificial Intelligence (cs.AI); Genomics (q-bio.GN) |
| Cite as: | arXiv:2602.17162 [cs.AI] |
| (or arXiv:2602.17162v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2602.17162 arXiv-issued DOI via DataCite |
Submission history
From: Ariel Larey [view email]
[v1]
Thu, 19 Feb 2026 08:20:51 UTC (824 KB)
[v2]
Mon, 25 May 2026 17:29:16 UTC (1,754 KB)
[v3]
Tue, 11 Aug 2026 09:21:39 UTC (1,746 KB)
[v4]
Thu, 15 Oct 2026 13:37:06 UTC (1,764 KB)
来源：arXiv cs.AI · arxiv.org