Getting training to work

From one update to training

In Meeting 4, we used backpropagation to calculate gradients, which help us choose changes to the weights and biases that reduce loss and help the model learn. In this lesson, we repeat that process automatically so the model can keep learning over many training steps.

Watch

3Blue1Brown · Gradient descent, how neural networks learn ↗

5:12–9:24

Then explore

Try different learning rates, see why an all-zero start gets stuck, and compare random starting distributions.

Learning rate

What is one training step?

One step is one complete loop of the process from Meeting 4: run the model forward, then use backpropagation to calculate the gradients. This demo averages the gradients across all four examples.

Multiply each gradient by the learning rate, then subtract it from its weight or bias. Updating all the weights and biases together completes one step.

New value = old value − learning rate × gradient.

Speed

Ready

Mean loss Starting loss

Loss over training steps

Hover or tap the graph to inspect a step. Arrow keys work too.