arXiv cs.MA· Jia Chen, Chao Huang, Xinrong Hu, Mingdi Huang, Ruyang Zhao, Jun Bai·· 11 小时前AI 评分20
GarmentSegAgent：基于文本自细化的网页时尚图像自主服装分割框架
GarmentSegAgent: Autonomous Garment Segmentation for Web Fashion Images via Textual Self-Refinement
AI 导读
GarmentSegAgent 是一个无需人工空间提示词、通过迭代文本自细化实现自主服装分割的框架，核心是 CAST（Constrained Attribution-guided Selective TextGrad）方法，可将视觉错误归因与定向文本优化连接起来，在推理时实现针对单张图像的纠错且保持基础模型冻结。
正文 · 原文
Abstract:Web fashion images support applications such as visual product search and consumer-to-shop matching, yet their visual heterogeneity makes reliable garment segmentation challenging. Despite advances in promptable segmentation, autonomously identifying and correcting garment-specific errors without human-provided spatial prompts remains difficult. We propose GarmentSegAgent, an autonomous framework that integrates semantic planning, spatial execution, and visual evaluation through iterative textual self-refinement. At its core, we introduce CAST (Constrained Attribution-guided Selective TextGrad), which connects visual error attribution with targeted textual optimization. CAST identifies instruction components responsible for segmentation errors, selectively updates them through textual gradients, and validates the resulting instructions against garment-specific constraints. This enables image-specific error correction at inference time while keeping the underlying foundation models frozen. Experiments on DeepFashion2, ModaNet, and Fashionpedia demonstrate consistent improvements in region overlap and boundary accuracy, alongside more reliable refinement than conventional TextGrad. By enabling visual feedback to guide controlled instruction updates, GarmentSegAgent offers a promising direction for autonomous garment understanding, with potential applications in scalable fashion content analysis, visual retrieval, and consumer-to-shop matching on the Web.
| Comments: | 12 pages |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2610.36506 [cs.CV] |
| (or arXiv:2610.36506v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36506 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jun Bai [view email]
[v1]
Tue, 27 Oct 2026 02:10:25 UTC (12,677 KB)
来源：arXiv cs.MA · arxiv.org