Deep Learning · MT2 · section
Sequence mappings and applications
Recurrent neural networks
Definition. A sequence-model configuration specifies how many inputs and outputs are processed and how their time indices relate. One-to-many, many-to-one and many-to-many configurations correspond to different prediction tasks.
Explanation and working
| Mapping | Meaning | Example and output |
|---|---|---|
| One to many | One input conditions several outputs | Image captioning: image to a word sequence |
| Many to one | Sequence summarized as one output | Review sentiment: words to positive neutral or negative |
| Many to many aligned | Output associated with each input time | Video frame labeling or per-word tagging |
| Many to many unaligned | Output sequence length may differ | Encoder-decoder translation |
Music generation can use earlier notes to predict the next note repeatedly. Next-word prediction uses the prefix to produce a probability distribution over a vocabulary. A word with the largest predicted probability is a simple choice, but the precise prediction cannot be calculated without the learned model's weights.
Sentence explanation: In “The dogs that bark near my house are loud,” the model should retain the plural subject across the intervening words when choosing “are”. A useful hidden representation should retain subject information while reading later words. This is an interpretation of what a trained model may learn, not a claim that an individual gate is guaranteed to represent grammar.
Sentiment model answer: tokenize the review, map tokens to numerical embeddings, process them with a vanilla RNN, use a final state or pooled sequence representation, then a dense layer and softmax over three sentiment classes. Train with categorical cross-entropy on labeled reviews; evaluate on held-out data. A bidirectional model is not needed for this answer and is not explicitly listed in the current syllabus.