# Formula reference

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/formulas

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/index/formulas/

1. [Mean, variance and standard deviation](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/1/)
2. [Standardization](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/2/)
3. [Power, constant and sum derivatives](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/3/)
4. [Chain rule](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/4/)
5. [Dot product and a neuron](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/5/)
6. [Matrix multiplication](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/6/)
7. [Layer forward pass](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/7/)
8. [Sigmoid and its derivative](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/8/)
9. [Tanh and its derivative](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/9/)
10. [ReLU and its derivative](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/10/)
11. [Leaky ReLU](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/11/)
12. [Stable softmax](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/12/)
13. [Squared and absolute losses](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/13/)
14. [Binary cross entropy](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/14/)
15. [Categorical cross entropy](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/15/)
16. [Output delta for half squared loss](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/16/)
17. [Sigmoid plus BCE; softmax plus CCE](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/17/)
18. [Hidden delta, weight gradient and bias gradient](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/18/)
19. [Gradient descent](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/19/)
20. [Batch and mini-batch gradient](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/20/)
21. [Momentum: signed velocity](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/21/)
22. [Nesterov: look-ahead gradient](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/22/)
23. [AdaGrad](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/23/)
24. [RMSProp](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/24/)
25. [Adam: both memories](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/25/)
26. [Adam: correction and update](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/26/)
27. [L1 penalty and update](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/27/)
28. [L2 penalty and update with a half](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/28/)
29. [Inverted dropout](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/29/)
30. [Convolution output dimensions](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/30/)
31. [Convolution value](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/31/)
32. [Max, min and average pooling](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/32/)
33. [Convolution parameter count](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/33/)
34. [Flatten and dense parameter count](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/34/)
35. [Convolution multiply-accumulate count](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/35/)
36. [Concatenation and residual addition](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/36/)
37. [Vanilla RNN](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/37/)
38. [Perceptron update](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/38/)
39. [BPTT hidden delta](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/39/)
40. [BPTT recurrent-weight gradient](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/40/)
41. [LSTM gates and candidate](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/41/)
42. [LSTM candidate and output gate](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/42/)
43. [LSTM cell and hidden state](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/43/)
44. [GRU gates, candidate and update](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/44/)
45. [GRU state under the retain-old convention](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/45/)
46. [Batch normalization and learned scale/shift](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/46/)
47. [Global average pooling](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/47/)
48. [Top-k error percentage](https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/48/)