USDCraft：用 LLM 程序化生成可仿真的铰接式 3D 资产

Demo

Method

Generation

New articulated assets from text or images

Without any input mesh, USDCraft writes an asset program from a prompt or a reference image. Pretrained LLMs bring broad knowledge of everyday objects, so new categories need no task-specific training. Every result below is the compiled, articulated USD asset — drag to orbit; parts move through their joint limits.

Upright piano

Image → asset

USDCraftLoading 3D

Backyard playground set

Image → asset

USDCraftLoading 3D

Manual wheelchair

Text → asset

Prompt

Create a complete, functionally usable, simulator-ready manual wheelchair with two rotating drive wheels, swiveling front casters, fold-up footrests, push rims, brakes, seat, backrest, and armrests.

USDCraftLoading 3D

Parcel locker bank

Text + image → asset

Prompt

Create a complete, functionally usable, simulator-ready parcel locker bank with eight differently sized independently hinged doors, handles or electronic latches, kiosk screen, stable enclosure, and clear retrieval access.

USDCraftLoading 3D

Deployable satellite antenna

Text → asset

Prompt

Create a complete, functionally articulated, simulator-ready deployable satellite high-gain antenna assembly with a spacecraft mounting pedestal, two-axis azimuth-and-elevation gimbal, a parabolic reflector divided into a rigid center dish and two foldable side petals, locking hinge housings, central feed horn, support struts, cable-routing brackets, hard stops, and launch-restraint latches. The antenna must fold into a compact stowed configuration and deploy into a coherent dish shape, while the deployed reflector can aim through both gimbal axes without colliding with the pedestal.

USDCraftLoading 3D

Manual pallet jack

Text → asset

Prompt

Create a complete, functionally usable, simulator-ready manual pallet jack with pivoting steering handle, rotating steer and load wheels, lifting fork mechanism, control lever, and clear pallet entry.

USDCraftLoading 3D

Explore the generated assets

16 categories · pick an asset, scrub its joints, download the USDZ

Drag to orbit · scroll to zoom

Input image credits

• Domestic sewing machine — eggrole, CC BY 2.0
• Glass infuser teapot — USDCraft evaluation set
• Microwave oven — USDCraft evaluation set
• Reflective traffic cone — Pryskyřník, Public domain
• Upright piano — pianomap, CC BY 2.0
• Backyard playground set — USDCraft evaluation set
• Camera tripod — David Adam Kess, CC BY-SA 4.0
• Cubby bookshelf with drawers — USDCraft evaluation set
• Emergency exit sign — frozenchipmunk, CC BY 2.0
• Laboratory centrifuge — Ivangiesen, CC0 1.0
• Parcel locker bank — PinTweaks, CC0 1.0
• Potted houseplant — Thomas Tolkien, CC BY 2.0
• Stocked supermarket shelf — USDCraft evaluation set

Reconstruction

Rebuilding real objects for simulation

For real-to-sim, USDCraft rebuilds each source mesh as a program, separating fused parts and completing missing ones. The input can be a static asset, a raw phone scan, or a single image lifted to a mesh.

Source meshLoading 3D

USDCraft–AstraLoading 3D

Views are linked — orbit either side to compare the source with the rebuilt, articulated asset.

Comparison with prior methods

Same Hunyuan3D 2.1 input for every method · paper results (USDCraft and Mini Workflow: round 1)

Loading 3D

Input · image + Hunyuan3D 2.1 mesh

No complete articulated outputpart geometry missing

ArtLLM

Loading 3D

SIMART3 parts · 2 joints

Loading 3D

Particulate9 parts · 8 joints

Loading 3D

Mini Workflow–Astra12 parts · 11 joints

Loading 3D

USDCraft–Astra15 parts · 14 joints

All six views share one camera — part colors show how each method splits the object into rigid parts; joints sweep through their limits.

Simulation

Loads into Isaac Sim, ready to run

Every compiled asset carries collision geometry, masses and joint limits, so it runs in Isaac Sim (PhysX) without manual fixes. Each clip shows the static asset, its dynamic articulation, and its collision geometry.

Real2Sim2Real

Policies trained only in simulation, deployed on real objects

We scan three real objects, rebuild them with USDCraft, collect 200 demonstrations per task in simulation, and train a Diffusion Policy that is deployed on the physical robot with no real-world data.

Open drawer

Press toaster lever

Turn on toaster

Success rate across asset methods

Paper Table · 20 trials per setting · ↓ sim-to-real drop · I/M: image/mesh input

BibTeX

@article{zhang2026usdcraft,
  title={USDCraft: Geometrically Grounded Programmatic Modeling of Articulated 3D Assets for Simulation},
  author={Zhang, Chuanrui and Yang, Zaijia and Wang, Duomin and Shi, Lu and Zhou, Daquan and Zhang, Ruihua and Wang, Ziwei},
  journal={arXiv preprint arXiv:2610.11322},
  year={2026}
}