arXiv cs.AI· Jiawei Xiao, Boya Wang, Longtian Qiu, Shan Ning, Xuming He·· 10 小时前AI 评分22
RaRe：面向开放词汇场景图生成的推理感知关系表示学习框架
Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation
AI 导读
RaRe 将 MLLM 生成的思维链（CoT）推理依据转化为结构化关系嵌入，直接优化推理依据衍生的表示空间以提升关系间可区分性。该框架先用自监督对比学习将推理依据嵌入与文本化关系描述对齐，再用面向特异性的强化学习目标改进推理依据生成。在 VG150 和 PSG 上，RaRe 相比现有基线提升了类别均衡识别效果，但学习带来的额外增益因数据集和候选框设置而异。
正文 · 原文
Abstract:Open-Vocabulary Scene Graph Generation (OV-SGG) requires models to recognize visual relationships beyond the training vocabulary, yet existing methods often rely on dataset-specific object--predicate co-occurrence patterns. Under long-tailed distributions and noisy object localization, such reliance leads to biased, entangled relation representations, limiting both rare-relation recognition and generalization to unseen predicates. To address these challenges, we propose RaRe, a reasoning-aware relational representation learning framework that transforms MLLM-generated chain-of-thought (CoT) rationales into structured relation embeddings. Rather than using CoT only as an intermediate reasoning trace for final prediction, RaRe directly optimizes the rationale-derived representation space to improve inter-relation discriminability. Specifically, RaRe first aligns rationale embeddings with textualized relation descriptions through self-supervised contrastive learning, and then refines rationale generation with a specificity-oriented reinforcement learning objective that encourages semantically object-pair-specific relational evidence. Experiments on VG150 and PSG demonstrate improved category-balanced recognition over existing baselines, while the additional gains from learning vary across datasets and proposal settings. Code available at: this https URL.
| Comments: | Accepted by NeurIPS 2026, codes are available at project page this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13496 [cs.CV] |
| (or arXiv:2611.13496v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13496 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Longtian Qiu [view email]
[v1]
Tue, 10 Nov 2026 03:08:22 UTC (7,652 KB)
来源：arXiv cs.AI · arxiv.org