Deep Learning · MT2 · section

AlexNet and the reason for its components

CNN architectures and transfer learning

Definition. AlexNet is a deep convolutional architecture introduced for ImageNet classification, conventionally described as five convolutional layers followed by three fully connected layers. Its historical significance is the effective combination of learned convolutional features, nonlinear activations, GPU training and regularization at large scale.

Architecture and information flow

Input image → convolution and ReLU stages → spatial pooling → fully connected representations → class scores and softmax. The original model used local response normalization in selected early stages and overlapping max pooling. Grouped computation across two GPUs was part of the original implementation. Exact sizes must follow the architecture specified in an examination question.

Diagram: Convolution extracts representations before the fully connected classification stages.. Labels: Input | 227×227×3 | Conv1 | 11×11, 96 | Pool | 3×3, s2 | Conv2 | 5×5, 256 | Pool | 3×3, s2 | Conv3 | 3×3, 384 | Conv4 | 3×3, 384 | Conv5 | 3×3, 256 | Pool + flat | 9216 | Dense | 4096 | Dense | 4096 | Output | 1000 | Read rows in order. Apply nonlinearities between learned layers.

Role of the components

ReLU supplies a nonlinear transformation and avoids sigmoid-style saturation for positive preactivations. Convolution uses local connectivity and spatial weight sharing. Pooling reduces spatial dimensions and increases tolerance to small local changes. Dropout regularizes the fully connected layers. Data augmentation supplies transformed training examples to reduce dependence on a fixed view. These components perform different jobs and should not be described as interchangeable methods of reducing overfitting.

Limitations and exam distinction

The dense layers contribute a large number of parameters; the model has substantial memory and computation requirements. AlexNet is much shallower than later residual architectures. Its original use of local response normalization is different from batch normalization. For a comparison answer, identify its sequential convolutional design, then contrast it with Inception's parallel branches and ResNet's residual addition.