跳到正文
arXiv cs.AI· Elly Akhoundi, Mathieu Lamarre, Thibault Goyallon, Patrick Anderson, \'Etienne Danvoye·· 4 小时前AI 评分38

FaceRiff:用几何条件实现数据高效统一的文本到 3D 人脸资产合成

FaceRiff: Data-Efficient Unified Text-to-3D Face Asset Synthesis via Geometric Conditioning

AI 导读

FaceRiff 提出一种统一、数据与参数高效的框架,可直接从文本在共享 UV 空间生成完整 3D 人脸资产(网格、albedo 与位移)。它基于预训练 SDXL 骨干,为每个任务加装 LoRA 适配器并以 UV 法线图作几何条件,仅用约 350 个身份的 2,049 次扫描、单 GPU 训练,每个任务更新不到 3% 的骨干参数,无需逐实例优化。

正文 · 原文

View PDF HTML (experimental)

Abstract:High-resolution facial datasets with consistent UV-mapped geometry, albedo, and physically based displacement are scarce, yet such assets are essential for photorealistic digital-human rendering. Existing 3D face generation methods are often slow, single-modality, or require large specialized datasets. Meanwhile, large image diffusion models encode strong facial priors but do not natively produce lighting-independent textures, pore-scale displacement, or UV-aligned geometry. We present FaceRiff, a unified, data- and parameter-efficient framework for generating complete 3D face assets (mesh, albedo, and displacement) directly from text in a shared UV space. FaceRiff adapts a pretrained SDXL backbone with task-specific LoRA adapters and uses UV normal maps as geometric conditioning: per-instance normals guide albedo and displacement synthesis, while a dataset-mean normal provides a canonical spatial anchor for mesh generation. Using only 2,049 scans from approximately 350 identities and training on a single GPU, FaceRiff updates less than 3% of the backbone parameters per task and produces render-ready 3D face assets without per-instance optimization. Compared with FaceG2E and DreamFace, FaceRiff produces greater per-prompt diversity while preserving prompt alignment and fixed-topology UV consistency, and additionally generates lighting-independent albedo suitable for PBR relighting and pore-scale displacement absent from prior text-driven baselines. A blind study with 45 industry professionals shows that FaceRiff achieves substantially higher perceived realism than DreamFace, approaching that of real captures. Overall, FaceRiff demonstrates that pretrained diffusion priors can be adapted to unified 3D face asset synthesis under limited data.
Comments: Accepted to SIGGRAPH Asia 2026 Conference Papers. 11 pages, 11 figures, 2 tables
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Graphics (cs.GR)
ACM classes: I.3.7; I.3.5; I.2.10
Cite as: arXiv:2611.21867 [cs.CV]
  (or arXiv:2611.21867v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.21867

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Elly Akhoundi [view email]
[v1] Mon, 16 Nov 2026 20:31:52 UTC (42,265 KB)

来源:arXiv cs.AI · arxiv.org