FineART：细粒度标注双臂操作轨迹数据集与视觉-语言-动作模型

Authors:Jade Choghari, Pepijn Kooijmans, Mansi Agarwal, Yusuf Umut Ciftci, Aseem Doriwala, Catherine Weaver, Mouli Sivapurapu, Kai Yang, Thomas Wolf, Jackson Lee, Pragna Mannam

View PDF HTML (experimental)

Abstract:Robots operating in real-world environments must often execute complex, multi-step bimanual tasks over long horizons rather than single, isolated actions. Current manipulation datasets provide limited support for this capability: although single-arm datasets reach hundreds of thousands of trajectories, they typically provide only one high-level instruction per episode, while existing bimanual datasets provide subtask annotations for only part of their recorded hours. We present FineART, a densely annotated bimanual manipulation dataset comprising 40,543 episodes (1,718 hours) and 533,913 subtasks across 151 tasks. We also introduce FineART-VLA, a vision-language-action policy that predicts its own next subtask to guide its actions. Mid-training on FineART's subtask annotations improves FineART-VLA's success at following spatial instructions from 32.0% to 100.0%. With step-by-step human subtask guidance, success on unseen long-horizon tasks increases from 16.0% to 76.0%. Furthermore, FineART-VLA matches baseline performance on a new robot with 10x less fine-tuning data and generalizes zero-shot to unseen tasks. We open-source the full dataset, model weights, and training code.

Comments:	26 pages. Code and model weights will be integrated into Hugging Face LeRobot this https URL
Subjects:	Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Cite as:	arXiv:2609.36416 [cs.RO]
	(or arXiv:2609.36416v3 [cs.RO] for this version)
	https://doi.org/10.48550/arXiv.2609.36416 arXiv-issued DOI via DataCite

Submission history

From: Jade Choghari [view email]
[v1] Tue, 29 Sep 2026 00:13:29 UTC (5,967 KB)
[v2] Wed, 30 Sep 2026 23:07:44 UTC (5,967 KB)
[v3] Mon, 5 Oct 2026 22:13:07 UTC (5,967 KB)