跳到正文
arXiv cs.AI· Qiang Chen, Xiao Wang, Qingquan Yang, Hao Si, Zikang Yan, Guosheng Xu, Jin Tang·· 10 小时前AI 评分22

FusionCLIP:面向聚变等离子体诊断的多模态对比预训练框架

FusionCLIP: Multimodal Contrastive Pretraining for Fusion Plasma Diagnostics

AI 导读

FusionCLIP 是一个多模态对比预训练框架,通过选择性视频编码器保留信息性视觉 token、基于查询的时间序列编码器按观测时间戳整合异步诊断信号,并用对比学习对齐两种模态。

正文 · 原文

View PDF HTML (experimental)

Abstract:Visible-light videos and multivariate diagnostic signals provide complementary observations of tokamak discharges. However, existing fusion pretraining methods primarily learn representations by reconstructing or predicting observations, leaving the correspondence between these modalities underexplored. Representation learning is further challenged by spatially localized plasma regions in video frames and substantial differences in diagnostic sampling rates. We propose FusionCLIP, a multimodal contrastive pretraining framework. A selective video encoder retains informative visual tokens, while a query-based time-series encoder integrates asynchronous diagnostic signals using their observation timestamps. Contrastive learning aligns representations of the two modalities. We construct the EAST Multimodal Diagnostics (EAST-MMD) dataset, comprising 1307 EAST discharges, and evaluate the model through independent fine-tuning on three tasks: disruption prediction, edge-localized mode (ELM) classification, and H98 regression. FusionCLIP achieves a Joint score of 89.07%, with absolute gains of 2.41% and 2.90% over the best competing baseline, and the in-domain contrastive pretraining baseline CLIP-Base, respectively. Ablations and attention visualizations further support the contributions of its key components.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.09493 [cs.CV]
  (or arXiv:2611.09493v2 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.09493

arXiv-issued DOI via DataCite

Submission history

From: Qiang Chen [view email]
[v1] Fri, 6 Nov 2026 14:14:41 UTC (7,563 KB)
[v2] Wed, 18 Nov 2026 11:20:14 UTC (8,047 KB)

来源:arXiv cs.AI · arxiv.org