arXiv cs.AI· Yuwei Sun, Yuxuan Yao, Hui Li, Weihao Yuan, Siyu Zhu·· 11 小时前AI 评分29
ReMoA：多模态扩散模型中的递归模块化自适应与稀疏潜在精炼
ReMoA: Recursive Modular Adaptation via Sparse Latent Refinement in Multimodal Diffusion Models
AI 导读
ReMoA 框架被提出并集成到扩散模型的联合注意力层中，通过多步递归迭代精炼视觉 token，并借助稀疏模块选择高效共享参数。在 ImageNet 上，DiT-XL 的 recall 从 0.57 提升至 0.62；SD3-medium 的 GenEval 总分从 68.33% 提升至 71.18%，且递归过程未引入额外参数。
正文 · 原文
Abstract:Diffusion models have achieved success in high-fidelity data synthesis, yet their capacity for more complex, structured reasoning such as text following remains constrained. While advances in language models have leveraged strategies such as latent reasoning and recursion to enhance text understanding capabilities, extending these to multimodal text-to-image generation tasks is challenging due to the continuous and non-discrete nature of visual tokens. To tackle this problem, we draw inspiration from modular human cognition and propose a Recursive Modular Adaptation (ReMoA) framework integrated into joint attention layers of conventional diffusion models, iteratively refining visual tokens over multiple recursion steps while efficiently sharing parameters via sparse selection of neural modules. Comprehensive evaluations on both class-conditioned ImageNet generation and text-to-image generation using the GenEval and DPG benchmarks demonstrate that recursive visual latent refinement contributes to richer object details and better-aligned outputs. Additionally, visual latent trajectory and routing analyses reveal that neural modules learn to specialize across diffusion timesteps, with each module's trajectories diverging at later, low-noise steps to support context-adaptive refinement. ReMoA increases recall for DiT-XL from 0.57 to 0.62 on ImageNet, and improves the overall GenEval score for SD3-medium from 68.33% to 71.18%, without introducing additional parameters through recursion.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.25299 [cs.CV] |
| (or arXiv:2604.25299v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.25299 arXiv-issued DOI via DataCite |
Submission history
From: Yuwei Sun [view email]
[v1]
Tue, 28 Apr 2026 07:09:18 UTC (5,791 KB)
[v2]
Wed, 4 Nov 2026 04:13:44 UTC (7,737 KB)
来源：arXiv cs.AI · arxiv.org