# AlexNet and the reason for its components

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/8/tee-c8-s0

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c8-s0/

**Definition.** AlexNet is a deep convolutional architecture introduced for ImageNet classification, conventionally described as five convolutional layers followed by three fully connected layers. Its historical significance is the effective combination of learned convolutional features, nonlinear activations, GPU training and regularization at large scale.

### Architecture and information flow

Input image → convolution and ReLU stages → spatial pooling → fully connected representations → class scores and softmax. The original model used local response normalization in selected early stages and overlapping max pooling. Grouped computation across two GPUs was part of the original implementation. Exact sizes must follow the architecture specified in an examination question.

![Convolution extracts representations before the fully connected classification stages.](https://deep-learning-study-room.vercel.app/assets/figures/alexnet.svg)

Diagram: Convolution extracts representations before the fully connected classification stages.. Labels: Input | 227×227×3 | Conv1 | 11×11, 96 | Pool | 3×3, s2 | Conv2 | 5×5, 256 | Pool | 3×3, s2 | Conv3 | 3×3, 384 | Conv4 | 3×3, 384 | Conv5 | 3×3, 256 | Pool + flat | 9216 | Dense | 4096 | Dense | 4096 | Output | 1000 | Read rows in order. Apply nonlinearities between learned layers.

Convolution extracts representations before the fully connected classification stages. Tap the diagram to enlarge.

### Role of the components

**ReLU** supplies a nonlinear transformation and avoids sigmoid-style saturation for positive preactivations. **Convolution** uses local connectivity and spatial weight sharing. **Pooling** reduces spatial dimensions and increases tolerance to small local changes. **Dropout** regularizes the fully connected layers. **Data augmentation** supplies transformed training examples to reduce dependence on a fixed view. These components perform different jobs and should not be described as interchangeable methods of reducing overfitting.

### Limitations and exam distinction

The dense layers contribute a large number of parameters; the model has substantial memory and computation requirements. AlexNet is much shallower than later residual architectures. Its original use of local response normalization is different from batch normalization. For a comparison answer, identify its sequential convolutional design, then contrast it with Inception's parallel branches and ResNet's residual addition.