Deep Learning · TEE · section

What you should understand before optimization

Neural-network foundations

You should be able to explain why a target and prediction differ, compute a weighted sum, choose an activation, measure a loss, find a one-neuron gradient and update a weight. If these steps are weak, repeat the first training example and the ReLU backprop example. They contain the entire calculation pattern that the next chapters extend.

Diagram: These four operations form the training loop used by the optimizers in the next chapter.. Labels: The four operations in one training step | The four operations in one training step | Repeat this sequence using the current parameters. | Predict | Forward pass | Measure error | Compute loss | Find | gradients | Backpropagation | Adjust | Optimizer | update | Calculate all gradients before replacing the old weights.