arXiv cs.AI· Weize Cai, Yongqi Dong, Zhida Shao, Zixin Fu, Xiaoxi Hu·· 4 小时前AI 评分22
Semantic Prism:基于显式语义-图像接口的生成式分割框架
Semantic Prism: Generative Segmentation with an Explicit Semantic-Image Interface
AI 导读
Semantic Prism 用扩散蒸馏骨干生成语义 RGB 图像,通过固定类别-颜色码本的距离定义接口 logits,在 Cityscapes 500 张验证集上达到 72.07% 平均 mIoU,比直接接口解码提升 11.39 个百分点。
正文 · 原文
Abstract:Conditional image generators can render segmentation maps as semantic RGB images, but color drift and boundary mixing can undermine direct decoding. We present Semantic Prism, a framework that combines an explicit semantic-image interface with hierarchical refinement for road-scene segmentation. A generator adapted from a diffusion-distilled backbone produces a semantic RGB image with a single denoising update per crop. Distances to a fixed class-color codebook define interface logits and pixelwise class probabilities. Hierarchical Generator Evidence Alignment (HGEA) combines spatially aligned multilevel features from the VAE encoder of the generator with the input image, generated semantic image, and interface probabilities to predict additive corrections to these logits. Contextual Interface Hierarchy Disagreement (C-IHD) combines pointwise uncertainty, local context, and interface-refinement disagreement in a fixed score for ranking pixel errors, without an auxiliary predictor or an additional network pass. On the 500-image Cityscapes validation set, Semantic Prism achieves 72.07% mean mIoU, improving on direct-interface decoding by 11.39 percentage points. Extensive capacity-matched ablations show higher mean mIoU for multilevel refinement than for flat and single-level alternatives. The Cityscapes-trained model transfers to ACDC Dataset, achieving 46.89% mIoU without target-domain adaptation. Beyond Cityscapes, on BDD100K dataset, a separately trained model achieves 62.22% mIoU on its full 1,000-image validation set. Across the 3 evaluated datasets, C-IHD improves pixel-error area under the AUPR over uncertainty derived from maximum softmax probability, with segmentation predictions held fixed for each comparison. In particular, on ACDC, AUPR increases from 0.6580 to 0.7557, demonstrating that Semantic Prism presents more effective error ranking under adverse-condition domain shift.
| Comments: | 16 pages, 4 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Image and Video Processing (eess.IV) |
| Cite as: | arXiv:2608.11537 [cs.CV] |
| (or arXiv:2608.11537v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2608.11537 arXiv-issued DOI via DataCite |
Submission history
From: Yongqi Dong [view email]
[v1]
Wed, 12 Aug 2026 01:00:04 UTC (6,564 KB)
[v2]
Tue, 17 Nov 2026 17:08:08 UTC (12,408 KB)
来源:arXiv cs.AI · arxiv.org