# Regularization and generalization

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/6

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/chapter/6/

Overfitting, L1/L2 penalties, dropout and early stopping: mechanisms, equations, comparisons and numerical applications.

1. [Recognizing overfitting and underfitting](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c6-s0/)
2. [L1 and L2 add a cost for large weights](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c6-s1/)
3. [Drill 30 Regularized objective](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s30/)
4. [Drill 31 L1 update on positive and negative weights](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s31/)
5. [Drill 32 L2 update](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s32/)
6. [Drill 47 Mixed backpropagation and L2](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s47/)
7. [Dropout removes activations during training](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c6-s3/)
8. [Early stopping uses the best validation point](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c6-s4/)
9. [Choosing a remedy](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c6-s6/)
10. [Inverted dropout during training and inference](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c6-s7/)
11. [Drill 33 Inverted dropout](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s33/)
12. [Drill 48 Mixed dropout and a momentum update](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s48/)
13. [F25 Q1c](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c11-s2/)
14. [R26 Q1c](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c12-s2/)
15. [F23 Q7b](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c13-s15/)
16. [R24 Q3b](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c14-s7/)
17. [F22 Q3a](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c15-s7/)
18. [F22 Q6b](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c15-s14/)
19. [R23 Q2b](https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c16-s6/)