Deep Learning · MT2 · section
Four ways to reuse a pretrained model
CNN architectures and transfer learning
Definition. A pretrained model has already learned parameters from an earlier training task. We can reuse its complete classifier, reuse its feature extractor, or use its learned weights as the starting point for further training. The choice determines which output labels are available and which parameters change.
First identify two parts. The backbone converts an input image into useful features. The head converts those features into the required prediction, such as class scores. A new task can reuse the backbone while replacing the head.
Yaad rakhne ka cue: Purane model ka poora answer use karna, sirf features lena, ya uske weights ko aage train karna—yeh teen decisions alag hain.
The faculty reference divides these choices into the following four uses. The two feature-extractor uses differ in whether features are saved separately or computed inside the new model.
| Use | What happens to the pretrained part? | What learns for the target task? | Concrete example |
|---|---|---|---|
| Direct classifier | Keep the complete model and its existing output head. Run it on new inputs. | Nothing is updated during ordinary prediction. | A model already trained to classify cats and dogs labels new cat/dog photographs. It can do this because the required labels match its existing outputs. |
| Standalone feature extractor | Run images through the fixed backbone and save the resulting feature vectors. A feature vector is an ordered list of numbers describing the image. | Train a separate classifier using these saved vectors. | Extract one vector per product photograph, then train a classifier to distinguish acceptable, scratched and cracked products. The extractor stays unchanged. |
| Integrated feature extractor | Place the pretrained backbone inside the new network and freeze it. Add a new head. The backbone calculates features whenever the new model runs. | Train the new head while the backbone's weights remain fixed. | Use a frozen image backbone followed by a new three-output head for acceptable, scratched and cracked products. The complete image-to-class pipeline is one model. |
| Weight initialization followed by fine-tuning | Start from the pretrained weights, then allow selected pretrained layers to update. Replace or adapt the head as required. | Train the new head and the unfrozen part of the backbone. | After first training a product-defect head, unfreeze the last backbone block and train it with a smaller learning rate to adapt to the factory's photographs. |
Why the existing class labels matter
A cat/dog head has two outputs with learned meanings: cat and dog. Giving it a photograph of a scratched product does not create a new output called “scratched.” For the product task, use an appropriate new head and train it on labeled product examples. Direct reuse of the original classifier is suitable only when its output meanings and preprocessing match the required task. When the task and data distribution are unchanged, this is ordinary inference rather than adaptation to a new task.
Standalone versus integrated features
Both methods can keep the backbone frozen. In the standalone method, feature vectors are calculated first and saved as a separate dataset; a later classifier reads that dataset. In the integrated method, each image passes through the backbone and head within one pipeline. Freezing means that the backbone parameters receive no training updates; it does not mean that the backbone stops performing calculations or that its stored weights disappear.
When transfer learning is useful
It is useful when labeled target data are limited and a pretrained model has learned representations relevant to the target task. It can reduce training time and the amount of target data needed. Improvement is not guaranteed: unrelated source features, unsuitable preprocessing or excessive fine-tuning on a small dataset can hurt performance. Choose the frozen or fine-tuned configuration using validation data, then evaluate the chosen model on the held-out test set.
Exam answer structure: define transfer learning → identify the backbone and head → explain the selected reuse mode → state which weights remain fixed and which learn → give a matching example and a limitation.