Deep Learning · MT2 · section

Shared weights and RNN applications

Recurrent neural networks

Definition. Temporal parameter sharing means that one recurrent transition function is reused at each sequence step. It permits variable sequence lengths without learning a separate transition matrix for each position.

Explanation and working

The RNN applies the same input and recurrent weights at every time step. The hidden state changes as new inputs arrive; the learned rule itself is reused. This is temporal weight sharing.

Diagram: The same learned rule combines previous state and new input at each time step.. Labels: Combine the current input and the previous state | Combine the current input and the previous state | The new state depends on both paths. | Previous state | h at time t − 1 | Current input | x at time t | Combine | Weighted sum + | bias | Activate | New state h at t | The same recurrent weight matrices are used again at the next time step.

For sentiment classification, a sequence of words leads to one sentiment label: many-to-one. For image captioning, an image representation conditions a sequence of words: one-to-many. These patterns describe how inputs and outputs are arranged, not separate update rules.