arXiv cs.CL· Yuchi Wang, Rongyao Fang, Weihao Yu, Xilin Wei, Tiancheng Gu, Dingkang Yang, Hang Zhang, Dayiheng Liu, Hongsheng Li, Shuai Bai·· 12 小时前AI 评分32
RefCraft：以网页开发为场景评测视觉参考条件下的智能体生成能力
RefCraft: Benchmarking Visual-Reference-Conditioned Agentic Generation through Web Development
AI 导读
研究者提出 RefCraft 基准，包含 300 个真实多模态网页开发任务，将自然语言指令与 1 至 5 个参考网页截图结合，要求智能体选择性解读参考并渲染、视觉检查、迭代优化输出。在 9 个前沿（多模态）大模型上，最强系统总分仅 52.12/100，表明视觉参考条件下的智能体生成在真实网页开发场景中仍是显著挑战。评测采用可独立验证的关键点及程序化与 VLM 评判相结合的方式。
正文 · 原文
Abstract:Beyond basic tool use and coding skills, effectively incorporating visual references into agentic generation is equally essential for applying AI agents to real-world productivity tasks. We study this crucial capability through webpage development, a practical and readily evaluable setting in which existing benchmarks remain largely idealized: they typically focus on reconstructing visual targets or generating webpages from textual instructions alone, overlooking the more common workflow of developing from visual examples. Specifically, we introduce RefCraft, a benchmark comprising 300 realistic multimodal webpage-development tasks that combine natural-language instructions with screenshots from one to five reference webpages. Agents must selectively interpret and utilize these references while rendering, visually inspecting, and iteratively refining their outputs. We evaluate the resulting open-ended artifacts using independently verifiable keypoints and complementary programmatic and VLM judges. Across nine frontier (M)LLMs, the strongest system achieves an overall score of only 52.12 out of 100, revealing that visual-reference-conditioned agentic generation remains a substantial challenge for current models in practical webpage-development settings.
| Comments: | Project page: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.26826 [cs.CL] |
| (or arXiv:2610.26826v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.26826 arXiv-issued DOI via DataCite |
Submission history
From: Yuchi Wang [view email]
[v1]
Mon, 19 Oct 2026 17:55:06 UTC (3,869 KB)
[v2]
Wed, 4 Nov 2026 16:45:09 UTC (3,869 KB)
来源：arXiv cs.CL · arxiv.org