跳到正文
arXiv cs.CL· Zichen Zhu, Dingye Liu, Nan Jiang, Danyang Zhang, Hang Zheng, Shijie Zheng, Situo Zhang, Kunyao Lan, Yingzi Ma, Sheng Jiang, Lu Chen, Kai Yu·· 10 小时前AI 评分34

SynthDCB:用世界蒸馏构建可扩展的设备控制基准

SynthDCB: World Distillation for Scalable Device Control Benchmarking

AI 导读

SynthDCB 是一个通过世界蒸馏(WD)构建的设备控制与环境感知基准,实例化 Home、Car、Phone 三类场景,含 114 个设备类别、1,166 个函数和 95 个事件,覆盖 5 类任务、32 个子类型。

正文 · 原文

Authors:Zichen Zhu, Dingye Liu, Nan Jiang, Danyang Zhang, Hang Zheng, Shijie Zheng, Situo Zhang, Kunyao Lan, Yingzi Ma, Sheng Jiang, Lu Chen, Kai Yu

View PDF HTML (experimental)

Abstract:Scaling device-control evaluation requires scaling both interactive worlds and the tasks agents must solve. We introduce SynthDCB, a benchmark for environment awareness and device control built through world distillation (WD). World distillation transforms knowledge of device behaviors, constraints, and environmental effects into modular executable components. A unified capability representation connects tool discovery, world execution, and task synthesis. SynthDCB instantiates Home, Car, and Phone with 114 device classes, 1,166 functions, and 95 events, supporting five task families with 32 subtypes. Explicit states and tool-call traces support task validation and agent evaluation. We evaluate 22 models on 2,560 test tasks; the best achieves 88.8% mean Pass@1. Construction experiments demonstrate token efficiency and expansion of task coverage. At the richest knowledge level, WD fully covers 22/23 properties, 19/29 functionalities, and 30/39 event requirements specified in a real RobotVacuum protocol. Furthermore, the 1,000 sampled tasks semantically cover 81.6% of the 854 reference instructions in the real-world corpus. SynthDCB connects an expanding space of real-to-sim executable capabilities to verifiable agent evaluation.
Comments: 71 pages, 9 figures, 22 tables. Project website: this https URL
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2611.18639 [cs.LG]
  (or arXiv:2611.18639v2 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.18639

arXiv-issued DOI via DataCite

Submission history

From: Zichen Zhu [view email]
[v1] Wed, 14 Oct 2026 12:03:38 UTC (2,180 KB)
[v2] Thu, 19 Nov 2026 09:22:58 UTC (2,553 KB)

来源:arXiv cs.CL · arxiv.org