arXiv cs.CL· Yixia Zhang, Girish A. Koushik, Xilu Wang, Wenwu Wang·· 23 小时前AI 评分29
TextSphere:通过共享超球面对齐与均匀性实现高效文本到图像生成
TextSphere: Efficient Text-to-Image Generation via Shared Hyperspherical Alignment and Uniformity
AI 导读
TextSphere 提出一种高效文本到图像生成框架,在 Sphere Encoder 基础上引入语言条件与跨模态对齐,通过 AdaLN 注入预训练 CLIP 文本嵌入,并提出 SSAU 目标在超球面上联合优化语义对齐与表示均匀性。
正文 · 原文
Abstract:Recent advances in text-to-image generation have substantially improved image quality, particularly through diffusion-based methods that require multiple inference steps. In contrast, Sphere Encoder offers an efficient generative framework capable of producing images with fewer forward steps. However, extending it to free-form text is nontrivial, because its class-conditioned design lacks cross-modal alignment. To address this challenge, we propose TextSphere, an efficient text-to-image generative framework that extends Sphere Encoder with language conditioning and cross-modal alignment. We incorporate pretrained CLIP text embeddings through adaptive layer normalization (AdaLN) to guide image generation while retaining the original spherical sampling mechanism. To bridge the cross-modal representation gap, we introduce a Spherical Semantic Alignment and Uniformity (SSAU) objective that jointly promotes semantic alignment and representation uniformity on the hypersphere. Experiments on CUB-200-2011, MS-COCO-2014, and Flickr30k demonstrate that TextSphere consistently reduces generation FID (gFID) over the text-conditioned baseline while achieving reconstruction FID (rFID) below 1.0. With only 4 and 6 sampling steps, TextSphere achieves lower gFID than the compared diffusion- and GAN-based models trained from scratch on CUB-200-2011 and MS-COCO-2014, respectively. We further observe that improved reconstruction fidelity does not necessarily yield better generation quality. In addition, we use Partial Least Squares Singular Value Decomposition (PLS-SVD) to analyze cross-modal alignment and modality gap, offering insight into the effectiveness of our approach.
| Comments: | 21 pages, 8 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.23155 [cs.CV] |
| (or arXiv:2611.23155v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23155 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yixia Zhang [view email]
[v1]
Tue, 17 Nov 2026 18:08:53 UTC (32,567 KB)
来源:arXiv cs.CL · arxiv.org