跳到正文
arXiv cs.AI· Zehao Gao, Jie Feng, Wenda Zhao, Junpeng Zhang, Ronghua Shang, Weisheng Dong·· 10 小时前AI 评分22

EmergeFuse:面向开放词汇语义分割的类别相对红外-可见光融合框架

Fuse to Recognize the Unseen: Class-Relative Infrared-Visible Fusion for Open-Vocabulary Semantic Segmentation

AI 导读

EmergeFuse 是一个面向开放词汇语义分割(OVSS)的红外-可见光融合框架,通过类别相对线索选择与文本对比图像校正,将热红外互补证据转化为未见类别的语义修正。在三个基准上,其未见类 mIoU 最高提升 4.755%,严格互补转换率提升 15.730%,同时保持有竞争力的融合质量。该方法针对 CLIP 类模型仅用 RGB 预训练、直接引入 IR 会破坏 RGB-语言对齐的问题。

正文 · 原文

View PDF HTML (experimental)

Abstract:Visible-infrared image fusion leverages complementary thermal cues to enhance visual perception, particularly under challenging illumination. However, such perceptual complementarity does not readily translate into semantic gains in open-vocabulary semantic segmentation (OVSS), where CLIP-based models are predominantly pretrained on RGB imagery. Directly introducing IR information may instead disrupt RGB-language alignment, turning promising thermal cues into semantic interference that undermines unseen-class recognition. Our pixel-level RGB-IR semantic analysis reveals that class-relevant thermal cues can correct certain RGB misclassifications while preserving reliable RGB predictions, motivating RGB-anchored, class-relative thermal cue selection. Based on this insight, we propose EmergeFuse, an infrared-visible fusion framework for OVSS that selectively converts complementary thermal evidence into unseen-class semantic corrections. Specifically, class-relative cue selection evaluates RGB-IR semantic responses and calibrated thermal evidence relative to RGB predictions, generating class-relative correction requests through competitive selection. Text-contrast image rectification then translates these selected requests into semantic rectification directions through text contrasts, guiding evidence-gated feature updates to realize complementary thermal evidence for unseen-class recognition. Experiments on three benchmarks demonstrate gains of up to 4.755% in unseen-class mIoU and 15.730% in strict complementary conversion rate over existing fusion methods, while maintaining competitive fusion quality.
Comments: 11 pages, 6 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24316 [cs.CV]
  (or arXiv:2611.24316v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.24316

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zehao Gao [view email]
[v1] Wed, 18 Nov 2026 12:22:54 UTC (11,078 KB)

来源:arXiv cs.AI · arxiv.org