arXiv cs.AI· Jiangtao Fan, Anish Jindal, Amir Atapour-Abarghouei·· 14 小时前AI 评分22
MCSTN：面向低质量可穿戴传感器数据的缺失条件时空融合人体活动识别框架
Missingness-Conditioned Spatio-Temporal Fusion for Human Activity Recognition from Low-Quality Wearable Sensor Data
AI 导读
研究者提出端到端框架 MCSTN，直接从不完整观测窗口及其掩码预测人体活动，无需先重建信号或估计噪声水平。在 PAMAP2、Opportunity、WISDM 上，不完美观测下平均 macro-F1 分别达 94.24%、91.67%、96.22%，各数据集均优于对比方法。模型参数量不足一百万，单样本推理延迟 3.78 ms。
正文 · 原文
Abstract:Body-worn sensor networks capture human activities through complementary views, where nodes, modalities, and axes partially describe the underlying motion. In real deployments, hardware faults, wireless dropouts, and loose attachments often cause entire channels or nodes to go missing, while environmental interference degrades the remaining signals. The common impute-then-recognize pipeline introduces reconstruction errors that cannot be observed or corrected at inference. We propose the Missingness-Conditioned Spatio-Temporal Network (MCSTN), an end-to-end framework for robust activity recognition under imperfect sensing. MCSTN has three components: a structured imperfection generator that simulates point, temporal-block, axis, modality, and node failures, plus noise at varying signal-to-noise levels; a mask-aware representation that conditions node embeddings, attention, and temporal aggregation on the availability mask; and a complete-to-imperfect multi-view learning scheme where an exponential-moving-average teacher guides multiple degraded student views through semantic consistency. The encoder models temporal and cross-feature dependencies in parallel: a bi-directional selective state-space branch captures long-range temporal dependencies, a transposed-attention branch models cross-channel correlations, and adaptive weights fuse their outputs. At inference, the generator, teacher, and consistency loss are removed, so MCSTN predicts directly from the observed window and its mask without reconstruction or noise-level estimation. On PAMAP2, Opportunity, and WISDM, MCSTN achieves average macro-F1 scores of 94.24\%, 91.67\%, and 96.22\% under imperfect observations, the best among all compared methods on each dataset, and the best in every failure mode on Opportunity. It requires fewer than one million parameters, with 3.78~ms inference latency per sample.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.00913 [cs.CV] |
| (or arXiv:2605.00913v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.00913 arXiv-issued DOI via DataCite |
Submission history
From: Jiangtao Fan [view email]
[v1]
Wed, 29 Apr 2026 10:55:41 UTC (9,626 KB)
[v2]
Mon, 26 Oct 2026 12:57:05 UTC (7,925 KB)
来源：arXiv cs.AI · arxiv.org