arXiv cs.AI· Farnaz Soleimani (LISSI), Abdelghani Chibani (LISSI), Yacine Amirat (LISSI), Ghazaleh Khodabandelou (LISSI)·· 3 天前AI 评分22
神经符号图 Transformer 实现多模态人类活动序列的分层意图识别
Neuro-Symbolic Graph Transformers for Hierarchical Intention Recognition from Multimodal Human Activity Sequences
AI 导读
研究者提出神经符号图 Transformer（NSGT），通过可学习拓扑偏置将人类推理图（HR-G）融入自注意力机制，并加入可微一阶逻辑（FOL）损失约束原子动作、复合活动、低层意图与高层意图四层本体一致性。
正文 · 原文
Abstract:Proactive human-robot collaboration requires autonomous systems to infer not only immediate actions but also the underlying hierarchical intentions governing human behavior. Current deep learning approaches predominantly focus on flat action classification, failing to capture the structural dependencies connecting kinematic motion to high-level goals. A Neuro-Symbolic Graph Transformer (NSGT) is proposed within this multimodal framework, a Human Reasoning Graph (HR-G) is integrated into the self-attention mechanism via learnable topological biases. Furthermore, the architecture is augmented with a differentiable First-Order Logic (FOL) loss, acting as a structural regularizer by which ontological consistency is enforced across four levels: atomic actions, composite activities, Low-Level Intentions, and High-Level Intentions. Evaluation is performed on the synthesized NTU-Hierarchy dataset, a four-level ontological extension of NTU RGB+D 120, wherein existing sequences were mapped into 119,383 complex hierarchical episodes (85 actions, 16 activities, 8 LLIs, 4 HLIs). To prevent circular supervision, strict isolation between label generation and FOL constraints was maintained. In the base configuration, 91.65% top-1 action accuracy is achieved by the NSGT. When the FOL loss is incorporated into the baseline, logic consistency is improved by 3.26 percentage points to 96.67%. Through the HR-G biases, a tunable trade-off between accuracy and structural coherence is established. In the primary consistency-maximizing configuration ($\lambda$ = 0.5), a logic consistency of 94.59% is reached and ontological contradictions are reduced by 33%, while a robust 89.90% top-1 action accuracy is retained. Finally, interpretable reasoning evidence is provided by the learned HR-G weights, wherein transition edges are strongly exploited ($\beta$ trans $\approx$ +2.6) without explicit supervision.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.18176 [cs.AI] |
| (or arXiv:2610.18176v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18176 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Farnaz SOLEIMANI [view email] [via CCSD proxy]
[v1]
Tue, 13 Oct 2026 11:20:51 UTC (1,615 KB)
来源：arXiv cs.AI · arxiv.org