arXiv cs.AI· Yuhan Li, Zhiyu Jin, Yifan Tong, Wenxiang Shang, Benlei Cui, Xuanhong Chen, Hangcheng Zhu, Bingbing Ni·· 3 天前AI 评分22
ShoeFit：面向虚拟鞋类试穿的新数据集与双图流 DiT 框架
ShoeFit: A New Dataset and Dual-image-stream DiT Framework for Virtual Footwear Try-On
AI 导读
研究团队提出虚拟鞋类试穿框架 ShoeFit，并发布多视角鞋类试穿数据集 MVShoes，包含 7305 组标注图像三元组。ShoeFit 采用双流 DiT 架构，通过带 3D 旋转位置嵌入的多视角条件注入缓解视角错位，并用 LayeredRefAttention 借助背景特征调制鞋类隐变量以抑制背景导致的颜色失真。该工作已被 NeurIPS 2025 接收。
正文 · 原文
Abstract:Virtual footwear try-on (VFTON), a critical yet underexplored area in virtual try-on (VTON), aims to synthesize faithful try-on results given diverse footwear and model images while maintaining 3D consistency and texture authenticity. Unlike conventional garment-focused VTON methods, VFTON presents unique challenges due to (1) Data Scarcity, which arises from the difficulty of perfectly matching product shoes with models wearing the identical ones, (2) Viewpoint Misalignment, where the target foot pose and source shoe views are always misaligned, leading to incomplete texture information and detail distortion, and (3) Background-induced Color Distortion, where complex material of footwear interacts with environmental lighting, causing unintended color contamination. To address these challenges, we introduce MVShoes, a multi-view shoe try-on dataset consisting of 7305 well-annotated image triplets, covering diverse footwear categories and challenging try-on scenarios. Furthermore, we propose a dual-stream DiT architecture, ShoeFit, designed to mitigate viewpoint misalignment through Multi-View Conditioning with 3D Rotary Position Embedding, and alleviate background-induced distortion using the LayeredRefAttention which leverages background features to modulate footwear latents. The proposed framework effectively decouples shoe appearance from environmental interferences while preserving high-quality texture detail through decoupled denoising and conditioning branches. Extensive quantitative and qualitative experiments demonstrate that our method substantially improves rendering fidelity and robustness under challenging real-world product shoes, establishing a new benchmark in high-fidelity footwear try-on synthesis.
| Comments: | Accepted at NeurIPS 2025. Project website: this https URL , including proposed multi-view footwear try-on dataset MVShoes-clean |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18325 [cs.CV] |
| (or arXiv:2610.18325v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18325 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuhan Li [view email]
[v1]
Tue, 13 Oct 2026 13:03:29 UTC (29,656 KB)
来源：arXiv cs.AI · arxiv.org