Forward-Forward 算法中 goodness 函数的选择性与形状设计研究

View PDF HTML (experimental)

Abstract:The Forward-Forward (FF) algorithm trains networks layer-by-layer using a local "goodness function," yet the choice of activity statistic remains an important design question. We systematically study how selectivity and activity shape affect FF learning, introducing selective goodness functions based on top-$k$ and entmax-weighted energy alongside shape-sensitive objectives based on normalized higher-order moments. In particular, we propose burstiness goodness, a parameter-free, scale-invariant excess-kurtosis objective that rewards heavy-tailed activity patterns. Our study examines goodness functions together with activation and label-injection choices across six small-image datasets, and uses controlled experiments to isolate their effects. In validation-tuned, five-seed Fashion-MNIST experiments with matched ReLU activation, standard label injection, and per-layer scoring, burstiness improves accuracy from $73.2\pm0.9\%$ to $88.6\pm0.4\%$ over sum-of-squares. On CIFAR-10, burstiness improves block-local ResNet-18 accuracy from $28.9\pm0.5\%$ to $46.8\pm2.5\%$ under the same separate per-block label pathway. These results show that activity-shape statistics can provide effective local learning signals across the tested settings. Code is available at this https URL.

Comments:	Accepted at NeurIPS 2026 (Main Track, poster). 16 pages, 1 figure
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
Cite as:	arXiv:2604.13081 [cs.LG]
	(or arXiv:2604.13081v3 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2604.13081 arXiv-issued DOI via DataCite

Submission history

From: Şuayp Talha Kocabay [view email]
[v1] Sat, 28 Mar 2026 23:11:21 UTC (24 KB)
[v2] Thu, 16 Apr 2026 20:55:27 UTC (30 KB)
[v3] Thu, 5 Nov 2026 18:59:03 UTC (27 KB)