Backpropagation is not magic and it is not new. It is the chain rule from first-year calculus, applied mechanically to a computational graph in reverse. This is what actually happens when a network learns: the two passes, a worked example in NumPy, why it runs backward instead of forward, where the gradients vanish and explode, and what the optimizer does with the numbers.
Backpropagation
-
Backpropagation: How Neural Networks Actually Learn