跳到正文
arXiv cs.CL· Yixia Zhang, Girish A. Koushik, Xilu Wang, Wenwu Wang·· 23 小时前AI 评分29

TextSphere:通过共享超球面对齐与均匀性实现高效文本到图像生成

TextSphere: Efficient Text-to-Image Generation via Shared Hyperspherical Alignment and Uniformity

AI 导读

TextSphere 提出一种高效文本到图像生成框架,在 Sphere Encoder 基础上引入语言条件与跨模态对齐,通过 AdaLN 注入预训练 CLIP 文本嵌入,并提出 SSAU 目标在超球面上联合优化语义对齐与表示均匀性。

正文 · 原文

View PDF HTML (experimental)

Abstract:Recent advances in text-to-image generation have substantially improved image quality, particularly through diffusion-based methods that require multiple inference steps. In contrast, Sphere Encoder offers an efficient generative framework capable of producing images with fewer forward steps. However, extending it to free-form text is nontrivial, because its class-conditioned design lacks cross-modal alignment. To address this challenge, we propose TextSphere, an efficient text-to-image generative framework that extends Sphere Encoder with language conditioning and cross-modal alignment. We incorporate pretrained CLIP text embeddings through adaptive layer normalization (AdaLN) to guide image generation while retaining the original spherical sampling mechanism. To bridge the cross-modal representation gap, we introduce a Spherical Semantic Alignment and Uniformity (SSAU) objective that jointly promotes semantic alignment and representation uniformity on the hypersphere. Experiments on CUB-200-2011, MS-COCO-2014, and Flickr30k demonstrate that TextSphere consistently reduces generation FID (gFID) over the text-conditioned baseline while achieving reconstruction FID (rFID) below 1.0. With only 4 and 6 sampling steps, TextSphere achieves lower gFID than the compared diffusion- and GAN-based models trained from scratch on CUB-200-2011 and MS-COCO-2014, respectively. We further observe that improved reconstruction fidelity does not necessarily yield better generation quality. In addition, we use Partial Least Squares Singular Value Decomposition (PLS-SVD) to analyze cross-modal alignment and modality gap, offering insight into the effectiveness of our approach.
Comments: 21 pages, 8 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
Cite as: arXiv:2611.23155 [cs.CV]
  (or arXiv:2611.23155v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.23155

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yixia Zhang [view email]
[v1] Tue, 17 Nov 2026 18:08:53 UTC (32,567 KB)

来源:arXiv cs.CL · arxiv.org