# R26 Q2b

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/12/tee-c12-s5

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c12-s5/

## Question

**Question (10 marks).** Explain AlexNet regularization and training improvements; explain Inception multiscale features and 1 × 1 convolutions; recommend an architecture for large-scale image classification with justification.

## Answer — step by step

AlexNet uses a sequence of convolutional layers followed by fully connected classification layers. Its influential design used ReLU activations, max pooling, dropout in the fully connected part, data augmentation and GPU training. ReLU's positive derivative supports learning compared with heavily saturated sigmoid hidden units. Augmentation supplies varied training views; dropout reduces co-adaptation; GPU computation makes a large network practical to train.

![The common classroom AlexNet layout contains five convolutions and three dense layers](https://deep-learning-study-room.vercel.app/assets/figures/alexnet.svg)

Diagram: The common classroom AlexNet layout contains five convolutions and three dense layers. Labels: Input | 227×227×3 | Conv1 | 11×11, 96 | Pool | 3×3, s2 | Conv2 | 5×5, 256 | Pool | 3×3, s2 | Conv3 | 3×3, 384 | Conv4 | 3×3, 384 | Conv5 | 3×3, 256 | Pool + flat | 9216 | Dense | 4096 | Dense | 4096 | Output | 1000 | Read rows in order. Apply nonlinearities between learned layers.

The common classroom AlexNet layout contains five convolutions and three dense layers Tap the diagram to enlarge.

The common 227-pixel classroom layout is: Conv 11 stride 4 with 96 filters; pool 3 stride 2; Conv 5 padding 2 with 256 filters; pool 3 stride 2; three Conv 3 padding 1 layers with 384,384,256 filters; pool 3 stride 2; dense 4096; dense 4096; output 1000. Nonlinearities are applied between learned layers. Exact padding, grouping and input conventions must follow the question: the original model used channel groups in some layers, so parameter counts differ from a fully connected-channel simplification.

AlexNet follows a mainly sequential feature-extraction path. An Inception module processes several spatial scales in parallel and concatenates them, often using 1 × 1 bottlenecks to reduce cost. Neither architecture is universally best; the task, resource budget and validation performance determine the choice.

An Inception module captures features at multiple scales in parallel. A typical module has a 1 × 1 branch, a 1 × 1 reduction followed by 3 × 3 convolution, a 1 × 1 reduction followed by 5 × 5 convolution, and a pooling branch followed by 1 × 1 projection. Padding and strides preserve compatible spatial sizes so branch outputs can be concatenated along the channel axis.

![Parallel branches preserve height and width then concatenate channels](https://deep-learning-study-room.vercel.app/assets/figures/inception.svg)

Diagram: Parallel branches preserve height and width then concatenate channels. Labels: Input | 1 × 1 | 1 × 1 reduce | 3 × 3 | 1 × 1 reduce | 5 × 5 | 3 × 3 pool | 1 × 1 | Concatenate channels | All branch height/width values must match.

Parallel branches preserve height and width then concatenate channels Tap the diagram to enlarge.

A 1 × 1 convolution mixes input channels at each position. With fewer output channels, it forms a bottleneck before an expensive larger kernel. ReLU following it adds nonlinearity. This reduces the number of input channels processed by 3 × 3 or 5 × 5 kernels while preserving useful learned combinations.

For H × W spatial size, Cin input channels and Cout output filters, a K × K convolution uses H × W × Cin × Cout × K² multiply-accumulates when the output spatial size stays H × W. A bottleneck changes the calculation to H × W × Cin × r + H × W × r × Cout × K². A multiply-accumulate is counted as one MAC here; some FLOP conventions count it as two operations. The benefit depends on the selected bottleneck width r. Compared with AlexNet's sequential kernels, Inception explicitly combines local and larger-scale features within one module.

**Recommendation:**  an Inception-family model is a defensible choice when varied object scales and computation efficiency matter, because parallel scales with bottlenecks provide diverse features at controlled cost. Compare held-out accuracy, latency and memory with available alternatives. This is a justified choice, not a claim that Inception always beats AlexNet on every dataset.