arXiv cs.AI· Gerard Donahue, Guven Gergerli, Ayush Gupta, Reza Ghoddoosian, Enna Sachdeva, Faizan Siddiqui·· 10 小时前AI 评分24
ARQ-VAE：面向时序动作分割的自适应残差量化记忆压缩方法
Adaptive Residual Quantization for Memory-Efficient Temporal Action Segmentation
AI 导读
研究者提出 ARQ-VAE，一种无需逐帧标注的紧凑特征记忆表示，通过对帧级视频特征学习离散残差量化并按帧自适应截断残差链，仅保留重建效果最佳的残差前缀。在 Breakfast 数据集上，该方法将训练集存储从原始 I3D 特征库的 28 GB 降至 10 MB，配合 ASFormer 在监督设置下取得 71.5 Edit 和 51.2 F1@50。
正文 · 原文
Abstract:Temporal action segmentation (TAS) is commonly trained on large banks of pre-extracted frame features, but storing and replaying those features becomes a practical bottleneck for long-form video, especially in incremental TAS where past-task data must remain available to mitigate catastrophic forgetting. Recent TAS replay and condensation methods reduce this burden, but they rely on dense frame-wise annotations and allocate storage through fixed segment-level structures, limiting their flexibility for long procedural videos whose frame complexity varies substantially over time. We introduce Adaptive Residual-Quantized Variational AutoEncoder (ARQ-VAE), a compact and label-free memory representation for TAS features that learns a discrete residual quantization of frame-level video features and then adaptively truncates the residual chain on a per-frame basis, retaining only the residual prefix that best reconstructs each frame feature. This yields a practical compact feature memory that does not depend on frame-wise labels and can therefore support multiple TAS settings with the same stored representation, while a lightweight fine-tuning stage further improves reconstruction quality for the truncated codes that are actually retained. Across standard TAS benchmarks, ARQ-VAE achieves a substantially improved storage-performance trade-off over prior replay and condensation baselines. On Breakfast, our final representation reduces training-set storage from 28 GB for the original I3D feature bank to 10 MB while retaining strong downstream segmentation performance; with ASFormer, it achieves 71.5 Edit and 51.2 F1@50 in the supervised setting. We further show that the same label-free memory remains effective for incremental and unsupervised TAS. We position ARQ-VAE as a practical memory representation for TAS and a promising direction for broader dataset condensation applications.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13262 [cs.CV] |
| (or arXiv:2611.13262v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13262 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Faizan Siddiqui [view email]
[v1]
Mon, 9 Nov 2026 21:31:31 UTC (882 KB)
来源：arXiv cs.AI · arxiv.org