arXiv cs.AI· Huajian Zeng, Chaohua Yao, Yuantai Zhang, Jiaqi Yang, Rolandos Alexandros Potamias, Xingxing Zuo·· 14 小时前AI 评分30
StableHand：面向第一人称视频世界空间双手运动估计的质量感知流匹配框架
StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video
AI 导读
StableHand 是一个质量感知的流匹配框架，从第一人称视频估计世界空间双手 4D 运动，通过四通道质量信号（双手腕全局平移与手指关节）调节生成过程。在 HOT3D 和 ARCTIC 两个第一人称基准上，其所有报告指标均达 SOTA，W-MPJPE 相比最强基线降低 20-25%，在遮挡严重的 ARCTIC 序列上提升最大。该工作已被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Recovering world space 4D motion of two interacting hands from egocentric video is a fundamental capability for supervising robot policy learning, where wrist trajectories track the end-effector and finger articulations specify the grasp pose. Two major challenges arise in this setting: hands frequently leave the camera view for extended periods due to head motion, and persistent hand-object interactions cause severe occlusions of one or both hands. Existing methods uniformly condition on noisy hand motion observations without accounting for their per-frame reliability, leading to substantial performance degradation. Our key insight is that accurate world space hand motion estimation is tightly coupled with the quality of per-frame hand observations. To this end, we decompose the quality of hand motion observations extracted from an off-the-shelf hand pose estimator into four channels: wrist global translation and finger articulations for both hands. We propose StableHand, a quality-aware flow-matching framework conditioned on these four-channel quality signals, which are predicted by a learned quality network. We naturally incorporate the quality signals into the flow-matching process through a per-channel forward schedule, a quality-adjusted velocity target, AdaLN modulation of the DiT denoiser, and a quality-aware ODE initialization. This unified generative process preserves high-quality observations while reconstructing unreliable ones using a learned bimanual motion prior. Experiments on HOT3D and ARCTIC, two egocentric benchmarks featuring long missing-hand spans and persistent hand-object occlusions, show that StableHand achieves state-of-the-art performance across all reported metrics, reducing W-MPJPE by 20-25% compared to the strongest baseline, with the largest gains on heavily occluded ARCTIC sequences.
| Comments: | Accepted by NeurIPS 2026. Project Page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.18553 [cs.CV] |
| (or arXiv:2605.18553v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.18553 arXiv-issued DOI via DataCite |
Submission history
From: Huajian Zeng [view email]
[v1]
Mon, 18 May 2026 15:33:15 UTC (5,503 KB)
[v2]
Mon, 26 Oct 2026 14:05:12 UTC (4,893 KB)
来源：arXiv cs.AI · arxiv.org