AIAI Club
← Lesson overview
Guided tutorialForward propagation

Build up one connection at a time.

Step 1 of 6

1 → 1

Meet the parts

Node
A circle that holds one number.
Input node
The left circle. Its number is the starting information.
Connection
A line that carries a number from one node to the next.
Weight
The multiplier attached to a connection.
Output node
The right circle. It holds the result of this tiny network.
Bias
A stored number the receiving node adds after multiplication.
ReLU
The rule: negative becomes 0; positive stays unchanged.

Explain

  • The input circle, or input node, starts with a number. That starting number is called the input.
  • The connection multiplies that number by a stored number called the weight.
  • The output circle, or output node, adds another stored number called the bias.
  • Bias test before ReLU: keeping the input and weight at 1 while changing bias from 0 to 1 to 2 changes the result from 1 to 2 to 3. ReLU does not change these positive results.
  • ReLU changes a negative result to 0 and leaves a positive result unchanged.
  • The number shown inside the last circle, or output node, is called the output.

Change

  1. Start with Input 1 = 1, weight = 1, and bias = 0. The output is 1.
  2. Change only the weight to 0, then 2. Watch the output change to 0, then 2.
  3. Press Reset. Keep Input 1 and weight at 1. Change only bias from 0 to 1, then 2. Predict each output before checking Notice.
  4. Press Reset again. Set bias to −2, so 1 × 1 − 2 = −1. Predict what ReLU will do, then check that the output becomes 0.

Notice

  1. Starting check: 1 × 1 + 0 = 1, so the output is 1.
  2. Weight check: weight 0 makes the output 0; weight 2 makes the output 2.
  3. Bias check: biases 0, 1, and 2 make outputs 1, 2, and 3. ReLU leaves all three positive results unchanged.
  4. Negative check: bias −2 makes −1, so ReLU changes the output to 0.