R25 Q7b · Transfer learning

Answer — step by step

Definition. Transfer learning reuses representations or parameters learned on a source task to support a target task. With a pretrained neural network, common strategies are fixed feature extraction and fine-tuning selected pretrained layers.

Explanation and working

Transfer learning reuses a model or representation learned on one task for a related new task. A pretrained CNN may already detect useful edges, textures and shapes, saving data and training effort.

Diagram: Start with learned image features. Replace the task-specific head, then choose which pretrained layers may change.. Labels: Reuse learned features, then adapt the output | Reuse learned features, then adapt the output | Example: start with an image model trained on a large dataset. | New images | Your smaller | labeled dataset | Feature extractor | Pretrained | convolutional layers | New head | Your classes | Feature extraction: freeze the pretrained layers. | Fine-tuning: unfreeze selected layers and train carefully. | Use validation data to choose settings. A small learning rate often helps | preserve useful features.

Full scenario. Build a face-attendance model with few images per student. Explain a training plan.

Step 1. Define and split the data. Collect varied, permitted images and separate training/validation/test by suitable independent captures. Keep duplicates out of different splits. Apply the pretrained model’s required input size and normalization.

Step 2. Choose a suitable pretrained backbone. For a course example, use VGG 16 or ResNet as an image-feature extractor. ImageNet-pretrained VGG 16 was trained on ImageNet, not automatically on a face-specific dataset. A face-trained model can be a more task-specific choice if available.

Step 3. Replace the classifier. Remove the original class head. Add a new classifier matching the required student classes, or an appropriate embedding-based recognition system if that is the chosen formulation. For the exam classification example, use a softmax head and cross-entropy.

Step 4. Freeze and train. Freeze the base weights and train the new head first. Use suitable augmentation, regularization and validation monitoring.

Step 5. Fine-tune if justified. Unfreeze selected later layers and train at a smaller learning rate. Early layers may remain frozen. Compare validation performance and restore the best checkpoint.

Step 6. Evaluate. Use the held-out test set once for the final estimate. Include unfamiliar faces and a rejection rule if describing a real attendance system; a closed-set classroom example alone does not solve unknown-person recognition.

Feature extraction keeps the backbone fixed. Fine-tuning changes some pretrained weights. Training the entire model from scratch is a third option when enough representative data and compute are available, but it is usually harder for the stated small-dataset scenario.

Early image layers commonly learn reusable edges and textures. Later layers become more specialized to source-task categories. “Earlier” means closer to the input, regardless of whether a drawing places those layers at the top or bottom.

These are starting strategies, not universal rules. Fine-tuning an early layer means updating its pretrained weights, not automatically learning it “from scratch”. With very little data, simply unfreezing early layers because the task differs can overfit. Choose changes using validation performance and apply the required preprocessing consistently.

Example. With 100 labelled microscopic images, an ImageNet model may provide reusable low-level patterns, but its animal/object classifier is inappropriate. Replace the head, test fixed representations and monitor validation before increasing the number of trainable layers. With a large representative image dataset, broader fine-tuning becomes a more reasonable candidate.

Yaad rakho: data kam toh kam weights hilao; task alag toh features ki suitability pehle dekho.