Deep Learning · MT2 · chapter

Regularization and generalization

Overfitting, L1/L2 penalties, dropout and early stopping: mechanisms, equations, comparisons and numerical applications.

1. Recognizing overfitting and underfitting
2. L1 and L2 add a cost for large weights
3. Drill 28 Regularized objective
4. Drill 29 L1 update on positive and negative weights
5. Drill 30 L2 update
6. Drill 41 Mixed backpropagation and L2
7. Dropout removes activations during training
8. Early stopping uses the best validation point
9. Choosing a remedy
10. Inverted dropout during training and inference
11. Drill 31 Inverted dropout
12. Drill 42 Mixed dropout and a momentum update
13. F25 Q1c
14. R26 Q1c
15. F23 Q7b
16. R24 Q3b
17. F22 Q3a
18. F22 Q6b
19. R23 Q2b