Deep Learning · MT2 · section
F22 Q1e
F22 selected MT2 practice
Question
Question (4 marks). Suggest and sketch an RNN architecture for image captioning.
Answer — step by step
Use a CNN encoder to map the image to a visual representation and a recurrent decoder (vanilla RNN) to generate words one at a time. Condition the decoder’s initial state or its inputs on image features. At each step, vocabulary softmax predicts the next word based on the image and earlier words. Train using correct next-word targets and cross-entropy; stop generation at an end token. This is a one-to-many mapping from one image to a caption sequence.