All sections in this chapter belong to the TEE preparation route. Learn vanilla RNNs first, then the shared-gradient calculation, then LSTM and GRU. Do not skip the gates or BPTT because the MT2 route ends earlier.
Begin with the vanilla state, then learn why the gated variants carry and update memory differently. Tap the diagram to enlarge.