跳到正文
arXiv cs.AI· Jiaju Ma, Ashvini Kumar Jindal, Chuhua Wang, Sumithra Bhakthavatsalam·· 10 小时前AI 评分38

MotionGraphicsBench:用程序化验证评估动效图形生成

MotionGraphicsBench: Evaluating Motion Graphics Generation with Programmatic Verification

AI 导读

研究者推出 MotionGraphicsBench,一个含 100 个实例的专家精选基准,用确定性、人类可读的验证程序评估 LLM 根据静态 SVG 和提示词生成动效的能力,而非依赖参考动画或 VLM 评判。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models can generate motion graphics from static images and natural language requests, but it remains difficult to evaluate how well models follow requests that combine spatial, appearance, and temporal requirements. We introduce MotionGraphicsBench, an expert-curated benchmark of 100 instances for animating an input static SVG with a motion edit prompt. By supplying all required visual elements, our editing task reflects a designer-like workflow and isolates motion generation from graphic generation. Derived from real-world Lottie motion designs, our benchmark contains diverse combinations of spatio-temporal animation behaviors. Each benchmark instance pairs its model input with a deterministic, human-readable verification program that accepts different valid implementations and reports which prompt requirements pass or fail, rather than comparing against a reference animation or using a VLM judge. We construct these programs with a multi-agent pipeline using human-labeled animations and synthesized test cases. On a balanced test set of 180 animations, our programs achieve 93.3% raw agreement with human judgments, outperforming frontier VLM judges by 4.4%-12.8% while achieving at least 90% recall for both human approvals and rejections. Across 47 configurations from 32 LLMs and 12 providers, pass@1 ranges from 5.0% to 68.0%. Across 15 models at their highest reasoning effort, additional attempts improve every model, but pass@32 still ranges from 38.0% to 95.0%. Our analysis of verification programs reveals that generated animations often fail to match reference poses, follow requested motion trajectories, or coordinate related events. We fine-tune Qwen3.8-27B with group relative policy optimization (GRPO) using our verification programs as rewards, improving pass@1 from 30.31% to 52.19% on held-out instances. Our project page is at this https URL.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.10592 [cs.CV]
  (or arXiv:2611.10592v2 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.10592

arXiv-issued DOI via DataCite

Submission history

From: Jiaju Ma [view email]
[v1] Sat, 7 Nov 2026 08:59:49 UTC (1,606 KB)
[v2] Wed, 18 Nov 2026 08:46:12 UTC (2,871 KB)

来源:arXiv cs.AI · arxiv.org