Backpropagation, step by step
OPTIONAL · GRADIENT ESTIMATE
Estimate a gradient with one small nudge
A gradient is a rate: change in loss ÷ change in parameter. Nudge one weight, compare the losses, and notice the direction. This is an estimate, not exact backpropagation.
Watch the agreed video excerpt ↗
3:06–8:30 · shared influences and backward nudges
Safe nonzero example
Inputs 1, 2 → target 1. Starting loss: . Positive ReLU sums keep this estimate visible.
Inputs 1, 2 → target 1. Starting loss: . Positive ReLU sums keep this estimate visible.
Meeting 3 used all-zero values for easy forward arithmetic. This separate introduction starts nonzero so a ReLU gate does not hide the gradient.
STEP 1 · BACKPROP EXPLAINED
Forward pass
Purple shows sensitivity. Orange shows changes, applied together at the end.
Colors: blue values, purple sensitivities/parameters, orange error/loss/change, green recomputed values.
Rate: 0.05 · each step averages all four examples
CURRENT · TOP
Forward pass
values lossBACKWARD
Combine and gate
sensitivitiesPREVIEW