Place-it-R1：用环境感知 MLLM 推理实现视频物体插入

View PDF HTML (experimental)

Abstract:Video object insertion is fundamental to video editing, yet existing diffusion methods often produce visually plausible but physically inconsistent results. We present Place-it-R1, an end-to-end framework for physically plausible video object insertion driven by environment-aware MLLM reasoning. Rather than treating reasoning as a generic text prompt, Place-it-R1 uses the MLLM to analyze the target environment, infer object-scene interactions, and determine where an insertion is physically valid. The resulting reasoning is translated into two complementary forms of guidance for video diffusion: semantic guidance that describes the intended physical interaction and spatial guidance that provides a valid insertion region in each frame. To further align generation with local physical realism, we introduce Spatial Direct Preference Optimization, which leverages an MLLM to rank generated candidates, and introduces a region-aware preference objective that explicitly localizes physical-violation penalties to the inserted-object region. Place-it-R1 further offers flexible and standard modes to trade off environment adaptation and scene preservation. Extensive experiments show that Place-it-R1 produces more physically coherent and visually natural insertions than state-of-the-art methods and achieves competitive results against commercial systems.

Comments:	this https URL
Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2603.06140 [cs.CV]
	(or arXiv:2603.06140v4 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2603.06140 arXiv-issued DOI via DataCite

Submission history

From: Bohai Gu [view email]
[v1] Fri, 6 Mar 2026 10:48:27 UTC (11,238 KB)
[v2] Wed, 5 Aug 2026 03:29:38 UTC (2,854 KB)
[v3] Thu, 22 Oct 2026 09:36:04 UTC (3,791 KB)
[v4] Fri, 30 Oct 2026 06:28:18 UTC (3,790 KB)