Split and Bridge（SnB）：用扩散桥接实现多模态生成

View PDF HTML (experimental)

Abstract:Visual data are often expressed through multiple complementary modalities (e.g., images and segmentation masks), each capturing different aspects of the same underlying structure. Most existing multimodal methods adopt a top-down paradigm, learning a shared latent variable through joint training from scratch, which limits scalability. We propose Split and Bridge (SnB), a bottom-up framework for multimodal generation that couples pretrained unimodal diffusion models at sampling time. Rather than learning a unified latent space, SnB generates coherent multimodal samples by following a shared diffusion trajectory up to a split point and then branching into modality-specific trajectories guided by a bridge module. This design enables plug-and-play reuse of powerful pretrained models without joint training, making it scalable and easily adaptable. Across standard benchmarks (PolyMNIST, CelebAMask-HQ) and more complex settings (PIE-Bench++, ROCO), SnB achieves a strong balance between sample quality and coherence. We show that SnB naturally extends to downstream zero-shot tasks such as interleaved co-generation, image editing, and style coupling/decoupling. Our results demonstrate that bottom-up generation provides a practical framework that complements existing top-down multimodal approaches. Code is available at this https URL

Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as:	arXiv:2611.10441 [cs.CV]
	(or arXiv:2611.10441v2 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2611.10441 arXiv-issued DOI via DataCite

Submission history

From: Ahmad Arrabi [view email]
[v1] Sat, 7 Nov 2026 03:41:27 UTC (38,375 KB)
[v2] Tue, 10 Nov 2026 05:18:50 UTC (38,367 KB)