arXiv cs.AI· Guanzhong Sun, Junyi Ma, Yuxuan Wu, Erhang Zhang, Chenyang Xu, Yanzi Miao, Hesheng Wang·· 10 小时前AI 评分20
TOMAS:面向第一视角手部预测的拓扑感知操作建模与自适应分段框架
TOMAS: Topology-Aware Operation Modeling and Adaptive Staging for Egocentric Hand Forecasting
AI 导读
TOMAS 是一个用于第一视角手部预测的拓扑感知操作建模与自适应分段框架,通过将手部骨骼结构与关节运动动态同物体语义、空间关系关联,联合推断交互目标与潜在操作类型。该框架引入自适应分段机制,将未来手部运动组织为带可学习时间边界的有序潜在阶段,并以阶段特定几何子目标引导轨迹细化。在多个公开基准上,TOMAS 在多维、多目标手部运动预测中取得 SOTA 性能。
正文 · 原文
Abstract:Forecasting future hand movements from egocentric videos plays an important role in understanding human intentions and hand-object interactions. However, different manipulation intentions can lead to substantially different hand trajectories even when interacting with the same object. Existing methods have not sufficiently explored the intrinsic relationships between articulated hand movements and object semantics to capture such operation-dependent motion patterns. Therefore, we propose TOMAS, a topology-aware operation modeling and adaptive staging framework for egocentric hand forecasting. Specifically, we develop a topology-aware hand-object reasoning mechanism that associates hand skeletal structures and articulation dynamics with object semantics and spatial relationships to jointly infer potential interaction targets and latent manipulation operations. The resulting object-operation representations provide interaction conditions for future hand trajectory prediction, with supervision adapted to the annotations available in each dataset. Furthermore, we introduce an adaptive staging mechanism that organizes future hand movements into ordered latent stages with learnable temporal boundaries. Stage-specific geometric subgoals are then incorporated to guide trajectory refinement according to the underlying manipulation patterns. Experimental results on multiple publicly available benchmarks demonstrate that TOMAS achieves state-of-the-art performance in multi-dimensional and multi-target hand motion forecasting.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23970 [cs.CV] |
| (or arXiv:2611.23970v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23970 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Guanzhong Sun [view email]
[v1]
Wed, 18 Nov 2026 07:10:44 UTC (38,145 KB)
来源:arXiv cs.AI · arxiv.org