arXiv cs.CL· Faris Alasmary·· 11 小时前AI 评分18
反向传播算法的详细推导：从链式法则到 NumPy 实现
Detailed Derivation of The Backpropagation Algorithm
AI 导读
一篇 41 页教程逐步推导了全连接前馈网络的反向传播算法，网络采用 sigmoid 激活函数和平方误差损失。推导从单个权重的链式法则求导出发，经误差项 δ 得到矩阵形式的四条通用反向传播方程，并扩展到 m 个训练样本与 mini-batch 梯度下降。每条方程均对应 NumPy 实现中的代码行，代码在 AHDBase 手写阿拉伯数字数据集上训练，完整代码随文提供。
正文 · 原文
Abstract:Backpropagation is the standard method for computing the gradients that train neural networks, but it is usually presented in a compact vector form that skips the intermediate steps. This tutorial derives backpropagation step by step for a fully connected feedforward network with sigmoid activations and a squared-error loss. Using a running example with three layers, we first differentiate the error with respect to a single weight by following its path through the network with the chain rule. We then show how summing over all paths leads to the error terms $\delta$ and to the four general backpropagation equations in matrix form. Every term is annotated with its dimensions, so each equation can be checked for consistency. The derivation is then extended to $m$ training examples, where the weight gradients become a single matrix product and the bias term corresponds to broadcasting. It is also extended to mini-batch gradient descent, including why the gradient of a mini-batch estimates the full gradient. Batch and mini-batch training are both given as algorithms. Finally, each equation is linked to the corresponding line of a NumPy implementation trained on the AHDBase dataset of handwritten Arabic digits, and the complete code is included.
| Comments: | 41 pages, 2 figures, 2 algorithms. Code: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.02152 [cs.CL] |
| (or arXiv:2611.02152v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02152 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Faris Alasmary [view email]
[v1]
Sun, 1 Nov 2026 19:35:16 UTC (23 KB)
来源：arXiv cs.CL · arxiv.org