arXiv cs.CL· Kasra Kamali, Basim Azam, Tuan Duong Trinh, Jean Honorio, Naveed Akhtar·· 11 小时前AI 评分29
ODC：面向文生图语义对齐的推理时正交漂移校正方法
ODC: Orthogonal Drift Correction for Improved Text-to-Image Semantic Alignment at Inference
AI 导读
研究者提出推理时引导方法 Orthogonal Drift Correction（ODC），无需重训练模型或额外用户输入即可缓解文生图语义漂移。ODC 先在共享视觉-语言嵌入空间定位初始图像与提示词的漂移向量，提取其与提示词方向正交的分量并经词汇表代理机制回译为文本，再通过自适应降秩概念移除模块生成精炼文本条件进行二次生成。
正文 · 原文
Abstract:Text-to-image models have achieved remarkable success in generating high-quality images from textual descriptions. However, they often struggle with "semantic drift," where the generated output fails to precisely align with complex or nuanced text prompts. While recent approaches have attempted to address semantic errors regarding attribute binding or object presence, there remains a gap for a more holistic method that addresses these issues by directly refining the text embeddings of the initial user prompt. In this work, we introduce Orthogonal Drift Correction (ODC), an inference-time guidance method designed to mitigate semantic drift without requiring model retraining or additional user inputs. ODC guides image generation through a two-stage process. In the first stage, it identifies the semantic drift by evaluating the initially generated image against the user prompt in a shared vision-language embedding space. It then isolates the component of this drift vector that is orthogonal to the prompt's direction and translates it back into text via a vocabulary-based surrogate mechanism. In the second stage, it produces refined text conditioning for a second generation pass by feeding both the initial text embedding and the re-embedded drift representation into an adaptive rank-reduced concept removal module. Our experiments demonstrate that ODC consistently improves prompt-image alignment, increasing CLIP scores by up to 5.4% over the baseline on the COALBench-300 and PartiPrompts benchmarks. Furthermore, on the T2I-CompBench complex subset, ODC improves the average 3-in-1 score by 4.5% over the competing methods. As a plug-and-play module, ODC offers a practical and efficient method for improving the reliability of text-to-image models. Code is available at this https URL.
| Comments: | Accepted to Transactions on Machine Learning Research (TMLR) 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.00819 [cs.CV] |
| (or arXiv:2611.00819v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00819 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kasra Kamali [view email]
[v1]
Sat, 31 Oct 2026 11:00:01 UTC (23,676 KB)
来源：arXiv cs.CL · arxiv.org