跳到正文
arXiv cs.AI· Malak Gamal El-Din, Yasser Shoukry, Salma Elmalaki·· 10 小时前AI 评分31

BeFit:将模型复杂度从权重转移到 Bernstein 激活,面向资源受限传感设备

BeFit: Shifting Complexity from Weights to Bernstein Activations for Resource-Constrained Sensing

AI 导读

BeFit 是一个端到端框架,通过可学习的 Bernstein 多项式激活把模型容量从权重转移到激活函数上,并结合硬件感知 NAS、知识蒸馏与无乘法权重投影。在人体活动识别任务上,同架构 BeCNN 平均比 ReLU CNN 高 2.75 个百分点;在 TI CC3220SF 上权重投影带来平均 2.2x 加速,仅损失 0.8 pp。

正文 · 原文

View PDF HTML (experimental)

Abstract:Time-series sensing increasingly relies on neural networks, yet deployment on resource-constrained microcontrollers remains challenging. Aggressive model compression can reduce accuracy, while restricting weights to multiplication-free representations introduces further losses. We present BeFit, an end-to-end framework that shifts model capacity from weights to learnable Bernstein polynomial activations. BeFit (i) finds compact Bernstein-activated convolutional neural networks (BeCNNs) through hardware-aware neural architecture search and trains them via knowledge distillation; (ii) projects their weights to multiplication-free codes, freezes the codes, and recovers the resulting accuracy loss through the learnable activations; and (iii) generates model-specific kernels that exploit the projected weights for multiplication-free convolutions. On human activity recognition, BeCNNs are 2.75 percentage points (pp) more accurate on average than ReLU CNNs of identical architecture. On a TI CC3220SF (Cortex-M4), weight projection speeds BeCNNs up by 2.2x on average while losing 0.8 pp, compared with 2.4 pp for ReLU CNNs. Against the MLPerf Tiny DS-CNN reference model running on TensorFlow Lite Micro with CMSIS-NN, the unprojected BeCNN achieves 1.0 pp higher accuracy, 2.2x lower latency, and 6.4x lower flash, while the projected model remains within 1.0 pp of the reference accuracy and runs 5.9x faster. In emulation on a RISC-V core without a hardware multiplier, the instruction-count reduction relative to the DS-CNN increases from 5.2x on the Cortex-M4 to 147x. BeFit also extends to transformers and MLPs, and across all tasks the final models stay within 1.56 pp of their float32 teachers on average.
Subjects: Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)
Cite as: arXiv:2611.23520 [cs.AI]
  (or arXiv:2611.23520v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.23520

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Malak Gamal El-Din [view email]
[v1] Tue, 17 Nov 2026 21:12:20 UTC (1,360 KB)

来源:arXiv cs.AI · arxiv.org