CARV：面向扩散模型教师梯度的方差缩减框架

View PDF HTML (experimental)

Abstract:Pretrained diffusion models serve as frozen teachers for pipelines such as text-to-3D, single-step distillation, and data attribution. The teacher gradients these pipelines consume are expectations over noise levels and Gaussian noise, estimated by Monte Carlo (MC). Each MC draw requires expensive upstream work (rendering, simulation, encoding), so estimator variance dominates compute cost. We introduce CARV, a compute-aware variance-accounting framework that casts these gradients as a classical two-stage sampling problem with asymmetric per-level costs. This motivates a hierarchical MC estimator that amortizes each expensive upstream computation by reusing it across cheap diffusion draws, with within-unit variance further reduced by timestep importance sampling (IS) and stratified inverse-CDF sampling. CARV delivers 2 to 3x effective compute multipliers on text-to-3D and data attribution without changing the objective. In text-to-3D, reuse provides most of the gain and IS with stratification adds about 25%. In single-step distillation, the same techniques cut gradient variance by an order of magnitude but do not improve downstream FID, marking the regime where MC variance is no longer the bottleneck.

Comments:	NeurIPS 2026 camera-ready. Project page: this https URL
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Computation (stat.CO); Machine Learning (stat.ML)
MSC classes:	65C05
ACM classes:	I.2.6; G.3
Cite as:	arXiv:2605.21489 [cs.LG]
	(or arXiv:2605.21489v3 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2605.21489 arXiv-issued DOI via DataCite

Submission history

From: Jonathan Lorraine [view email]
[v1] Wed, 20 May 2026 17:59:52 UTC (6,587 KB)
[v2] Thu, 21 May 2026 21:50:17 UTC (6,531 KB)
[v3] Mon, 9 Nov 2026 23:33:34 UTC (6,884 KB)