Deep Learning · TEE · chapter

Recurrent neural networks

State equations, parameter sharing and sequence mappings. The MT2 route covers introductory RNNs; TEE adds recurrent training and gated architectures.

  1. Complete the entire sequence chapter
  2. What makes a sequence different
  3. Reading recurrent memory and gates
  4. The vanilla RNN equations from scratch
  5. A complete vector RNN calculation
  6. Drill 41 Two RNN steps
  7. How an RNN is trained
  8. BPTT with every arithmetic step visible
  9. Drill 45 BPTT shared weights
  10. Vanishing and exploding gradients
  11. LSTM a memory path with three gates
  12. Reading an LSTM equation one operation at a time
  13. LSTM complete numerical with supplied gates
  14. Drill 42 LSTM with a negative candidate
  15. LSTM numerical when gates must first be calculated
  16. Drill 43 Calculate gates before the cell
  17. Drill 50 Three step LSTM retention and replacement
  18. GRU two gates and one state
  19. Drill 44 GRU reset then mix
  20. Comparing the three recurrent models
  21. Sequence mappings and applications
  22. Recurrent networks: the complete relationship
  23. Gate logic without memorizing a story
  24. F25 Q1d
  25. F25 Q4b
  26. F25 Q5a
  27. R26 Q1d
  28. R26 Q4b
  29. R26 Q5a
  30. F23 Q1c
  31. F23 Q4b
  32. F23 Q6a
  33. F23 Q7a
  34. R24 Q5b
  35. R24 Q6a
  36. R24 Q7a
  37. R24 Q7b
  38. F22 Q1e
  39. F22 Q2a
  40. F22 Q5a
  41. F22 Q7a
  42. R23 Q1a
  43. R23 Q3a
  44. R23 Q5a
  45. R23 Q7a