arXiv cs.AI· Wentao Ye, Yali Ye, Zhiqing Xiao, Ru Peng, Xinpeng Ti, Liyao Li, Zhanming Shen, Peng Lu, Jie Zhang, Haobo Wang·· 11 小时前AI 评分34
StruPO：从结构反馈到提示词策略，学习忠实的文本到图像提示词编辑器
From Structural Feedback to Prompt Policies: Learning Faithful Text-to-Image Prompt Editors
AI 导读
StruPO 是一个结构感知的提示词编辑器，以校准后的场景图作为策略条件与图像监督的共享表示，通过删除、重排、插入实现 token 对齐的提示词修改。在三个 2K 提示词测试集和四个冻结生成骨干上，它改善了忠实度与美学的权衡，关系保真度持续提升，并获得 15 名标注者研究支持。该策略在图校准后一次前向即可产出可复用的优化文本，无需迭代式批评—重写循环。
正文 · 原文
Abstract:Prompt optimization should improve the image a user receives without changing the scene the user requested. Yet stylistic rewriting can weaken object attributes and relations even when global text--image similarity remains high. We propose StruPO, a structure-aware prompt editor that uses a calibrated scene graph as a shared representation for policy conditioning and image-grounded supervision. The graph exposes what should be preserved, while deletion, reordering, and insertion provide a token-aligned way to modify the prompt. This turns structural feedback from an external critique into a training signal for a reusable editing policy. Learning from that signal requires handling both uncertain grounding and stochastic image generation. StruPO retains grounding confidence through smooth reward shaping and averages multiple images for each edit before computing group-relative advantages. Experiments span three disjoint 2K-prompt test sets and four frozen generation backbones, with comparisons to learned prompt optimizers, iterative structural feedback, diffusion-model fine-tuning, and inference-time attention control. StruPO improves the faithfulness--aesthetics trade-off, with consistent gains in relational fidelity. Independent automatic evaluators and a 15-annotator study provide complementary evidence, and human preference margins increase with relation complexity. Controlled ablations connect these gains to graph conditioning, local editing, and image-grounded learning. The resulting policy produces reusable optimized text in one pass after graph calibration, without an iterative critique--rewrite loop.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36146 [cs.AI] |
| (or arXiv:2610.36146v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36146 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wentao Ye [view email]
[v1]
Mon, 26 Oct 2026 18:50:50 UTC (9,030 KB)
来源：arXiv cs.AI · arxiv.org