arXiv cs.AI· Rit Gangopadhyay, Alex Wong·· 13 小时前AI 评分31
DEX：让视觉基础模型适配鱼眼相机的深度估计与开放词汇分割
From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation
AI 导读
研究者提出 Distortion Extenders（DEX），一组可学习参数，通过自监督对齐损失把鱼眼图像的潜空间嵌入变换为接近透视图像的形式，从而将视觉基础模型泛化到鱼眼相机。DEX 与架构和任务无关，在单目深度估计和开放词汇分割上、对卷积和 Transformer 架构均持续优于基线，覆盖室内外鱼眼数据集。其激活还可解码为畸变系数以支持相机标定，代码已开源。
正文 · 原文
Abstract:Vision foundation models are capable of generalizing across 3-dimensional (3D) scenes with high-fidelity estimates; their empirical success can be attributed to training on large-scale datasets of perspective images. However, when transferred to wide field-of-view (FoV) images, such as those captured by fisheye cameras, they return erroneous outputs due to a covariate shift stemming from the radial distortion on the image pixels. We propose a method to generalize vision foundation models to fisheye cameras. The crux of our method lies in a set of learnable parameters, termed Distortion Extenders (DEX), that model the fisheye distortion coefficients and the distributional shift between fisheye and perspective images encoded in the latent space. By minimizing a self-supervised alignment loss, DEX transforms the latent embeddings of fisheye images to resemble those of perspective images to recover high-fidelity estimates. DEX is architecture- and task-agnostic: We demonstrate DEX on monocular depth estimation and open-vocabulary segmentation for convolution- and Transformer-based architectures, where we consistently improve over baselines across indoor and outdoor fisheye datasets. As a byproduct, the activations of DEX can also be decoded to distortion coefficients to support camera calibration. Code available at: this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.27860 [cs.CV] |
| (or arXiv:2608.27860v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2608.27860 arXiv-issued DOI via DataCite |
|
| Related DOI: | https://doi.org/10.1007/978-3-032-37627-5_14
DOI(s) linking to related resources |
Submission history
From: Suchisrit Gangopadhyay [view email]
[v1]
Fri, 28 Aug 2026 03:10:18 UTC (30,430 KB)
[v2]
Mon, 26 Oct 2026 21:50:37 UTC (15,270 KB)
来源：arXiv cs.AI · arxiv.org