Deep Learning · MT2 · section

R26 Q6a

R26 selected MT2 practice

Question

Question (10 marks). For the binary CNN below, calculate output dimensions and trainable parameters after every layer. Input 256 × 256 × 3. → Conv K = 5, filters = 32, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Conv K = 5, filters = 64, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Conv K = 5, filters = 128, stride = 1, padding = 1. → Pool K = 2, stride = 2, padding = 0. → Flatten. → Dense 84 units. → Dense 32 units. → Dense 1 units. The printed output size is unspecified; assume one sigmoid output. Unspecified convolution padding is taken as valid.

Answer — step by step

Given

Input 256 × 256 × 3.

Conv K = 5, filters = 32, stride = 1, padding = 0.

Pool K = 2, stride = 2, padding = 0.

Conv K = 5, filters = 64, stride = 1, padding = 0.

Pool K = 2, stride = 2, padding = 0.

Conv K = 5, filters = 128, stride = 1, padding = 1.

Pool K = 2, stride = 2, padding = 0.

Dense 84 units.

Dense 32 units.

Dense 1 units.

assume one sigmoid output.

To find

Find every layer shape, its parameter count and the network total.

Formula — which one and why

Complete CNN calculation. Carry each output shape into the next layer. Convolution changes spatial size and channel count; pooling changes spatial size; flattening converts all activations into dense-layer inputs.

H,W,C = feature-map height, width and channels; K = kernel size; S = stride; P_pad = padding. One bias is assumed for each learned filter or dense neuron unless specified otherwise.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1
P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}LaTeX: P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}
n_{\mathrm{flat}}=HWCLaTeX: n_{\mathrm{flat}}=HWC
P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}LaTeX: P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}
P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0LaTeX: P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0

Yaad rakhne ka cue: Ek layer ka output, next layer ka input; shape aur parameters alag count.

Solution — step by step

Use output size \left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: \left\lfloor\frac{N+2P-K}{S}\right\rfloor+1, Conv weights K² × Cin × Cout and biases Cout, Dense weights Nin × Nout and biases Nout. Assume a bias per learned filter/unit, full input-channel connectivity and no trainable pooling parameters. ReLU and softmax activations themselves add no trainable parameters.

Layer-by-layer calculation

Carry the complete output shape of one layer into the next. Spatial size determines the number of activations; it does not multiply the number of shared convolution parameters.

Layer 1 — convolution

Layer 1: what changes and what is learned
Input shape256 × 256 × 3height × width × channels
Convolution32 filters · 5 × 5 · S = 1 · P = 0
Output shape252 × 252 × 32height × width × channels
Weights2,400
Biases32
Trainable parameters2,432

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 256 × 256 × 3. Use 32 filters of spatial size 5 × 5, stride 1, and padding 0 on each side.

1. Calculate height and width.

N_{\mathrm{out}}=\left\lfloor\frac{256+2(0)-5}{1}\right\rfloor+1=252LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{256+2(0)-5}{1}\right\rfloor+1=252

2. Determine output channels. Each filter produces one feature map. Therefore the output is 252 × 252 × 32.

3. Count trainable values. Each filter receives all 3 input channels. Its weights are reused at every spatial position.

N_{\mathrm{weights}}=5\times5\times3\times32=2{,}400LaTeX: N_{\mathrm{weights}}=5\times5\times3\times32=2{,}400
N_{\mathrm{biases}}=32LaTeX: N_{\mathrm{biases}}=32
P_1=2{,}400+32=2{,}432LaTeX: P_1=2{,}400+32=2{,}432

Layer 2 — pooling

Layer 2: what changes and what is learned
Input shape252 × 252 × 32height × width × channels
Pooling2 × 2 window · S = 2
Output shape126 × 126 × 32height × width × channels
Weights0
Biases0
Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 252 × 252 × 32. Use a 2 × 2 window, stride 2, and zero padding.

N_{\mathrm{out}}=\left\lfloor\frac{252-2}{2}\right\rfloor+1=126LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{252-2}{2}\right\rfloor+1=126

Pooling acts independently on each channel. The output is 126 × 126 × 32.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

Layer 3 — convolution

Layer 3: what changes and what is learned
Input shape126 × 126 × 32height × width × channels
Convolution64 filters · 5 × 5 · S = 1 · P = 0
Output shape122 × 122 × 64height × width × channels
Weights51,200
Biases64
Trainable parameters51,264

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 126 × 126 × 32. Use 64 filters of spatial size 5 × 5, stride 1, and padding 0 on each side.

1. Calculate height and width.

N_{\mathrm{out}}=\left\lfloor\frac{126+2(0)-5}{1}\right\rfloor+1=122LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{126+2(0)-5}{1}\right\rfloor+1=122

2. Determine output channels. Each filter produces one feature map. Therefore the output is 122 × 122 × 64.

3. Count trainable values. Each filter receives all 32 input channels. Its weights are reused at every spatial position.

N_{\mathrm{weights}}=5\times5\times32\times64=51{,}200LaTeX: N_{\mathrm{weights}}=5\times5\times32\times64=51{,}200
N_{\mathrm{biases}}=64LaTeX: N_{\mathrm{biases}}=64
P_3=51{,}200+64=51{,}264LaTeX: P_3=51{,}200+64=51{,}264

Layer 4 — pooling

Layer 4: what changes and what is learned
Input shape122 × 122 × 64height × width × channels
Pooling2 × 2 window · S = 2
Output shape61 × 61 × 64height × width × channels
Weights0
Biases0
Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 122 × 122 × 64. Use a 2 × 2 window, stride 2, and zero padding.

N_{\mathrm{out}}=\left\lfloor\frac{122-2}{2}\right\rfloor+1=61LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{122-2}{2}\right\rfloor+1=61

Pooling acts independently on each channel. The output is 61 × 61 × 64.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

Layer 5 — convolution

Layer 5: what changes and what is learned
Input shape61 × 61 × 64height × width × channels
Convolution128 filters · 5 × 5 · S = 1 · P = 1
Output shape59 × 59 × 128height × width × channels
Weights204,800
Biases128
Trainable parameters204,928

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 61 × 61 × 64. Use 128 filters of spatial size 5 × 5, stride 1, and padding 1 on each side.

1. Calculate height and width.

N_{\mathrm{out}}=\left\lfloor\frac{61+2(1)-5}{1}\right\rfloor+1=59LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{61+2(1)-5}{1}\right\rfloor+1=59

2. Determine output channels. Each filter produces one feature map. Therefore the output is 59 × 59 × 128.

3. Count trainable values. Each filter receives all 64 input channels. Its weights are reused at every spatial position.

N_{\mathrm{weights}}=5\times5\times64\times128=204{,}800LaTeX: N_{\mathrm{weights}}=5\times5\times64\times128=204{,}800
N_{\mathrm{biases}}=128LaTeX: N_{\mathrm{biases}}=128
P_5=204{,}800+128=204{,}928LaTeX: P_5=204{,}800+128=204{,}928

Layer 6 — pooling

Layer 6: what changes and what is learned
Input shape59 × 59 × 128height × width × channels
Pooling2 × 2 window · S = 2
Output shape29 × 29 × 128height × width × channels
Weights0
Biases0
Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 59 × 59 × 128. Use a 2 × 2 window, stride 2, and zero padding.

N_{\mathrm{out}}=\left\lfloor\frac{59-2}{2}\right\rfloor+1=29LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{59-2}{2}\right\rfloor+1=29

Pooling acts independently on each channel. The output is 29 × 29 × 128.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

The division gives 28.5; take its floor before adding 1. A partial window outside the unpadded input is not included.

Layer 7 — flatten

Layer 7: what changes and what is learned
Input shape29 × 29 × 128height × width × channels
FlattenRearrange activations
Output shape107,648output features
Weights0
Biases0
Trainable parameters0

The same activation values are rearranged into one vector. No parameter is learned.

Input: 29 × 29 × 128. Rearrange all activation values into one vector.

n_{\mathrm{flat}}=29\times29\times128=107{,}648LaTeX: n_{\mathrm{flat}}=29\times29\times128=107{,}648

Output: a vector with 107,648 entries. Flattening changes the arrangement, so it adds 0 trainable parameters.

Layer 8 — fully connected layer

Layer 8: what changes and what is learned
Input shape107,648input features
Dense layer84 neurons
Output shape84output features
Weights9,042,432
Biases84
Trainable parameters9,042,516

Every output neuron receives every input feature, with one bias per output neuron.

Input: 107,648 features. Output: 84 neurons. Every output neuron connects to every input feature and has one bias.

N_{\mathrm{weights}}=107{,}648\times84=9{,}042{,}432LaTeX: N_{\mathrm{weights}}=107{,}648\times84=9{,}042{,}432
N_{\mathrm{biases}}=84LaTeX: N_{\mathrm{biases}}=84
P_8=9{,}042{,}432+84=9{,}042{,}516LaTeX: P_8=9{,}042{,}432+84=9{,}042{,}516

Layer 9 — fully connected layer

Layer 9: what changes and what is learned
Input shape84input features
Dense layer32 neurons
Output shape32output features
Weights2,688
Biases32
Trainable parameters2,720

Every output neuron receives every input feature, with one bias per output neuron.

Input: 84 features. Output: 32 neurons. Every output neuron connects to every input feature and has one bias.

N_{\mathrm{weights}}=84\times32=2{,}688LaTeX: N_{\mathrm{weights}}=84\times32=2{,}688
N_{\mathrm{biases}}=32LaTeX: N_{\mathrm{biases}}=32
P_9=2{,}688+32=2{,}720LaTeX: P_9=2{,}688+32=2{,}720

Layer 10 — fully connected layer

Layer 10: what changes and what is learned
Input shape32input features
Dense layer1 neurons
Output shape1output features
Weights32
Biases1
Trainable parameters33

Every output neuron receives every input feature, with one bias per output neuron.

Input: 32 features. Output: 1 neurons. Every output neuron connects to every input feature and has one bias.

N_{\mathrm{weights}}=32\times1=32LaTeX: N_{\mathrm{weights}}=32\times1=32
N_{\mathrm{biases}}=1LaTeX: N_{\mathrm{biases}}=1
P_10=32+1=33LaTeX: P_10=32+1=33

Summary of the complete network

LayerSpatial or flatten calculationOutputWeightsBiasesTotal
1 conv\left\lfloor\frac{256+2 \times 0-5}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{256+2 \times 0-5}{1}\right\rfloor+1252 x 252 x 322,400322,432
2 pool\left\lfloor\frac{252-2}{2}\right\rfloor+1LaTeX: \left\lfloor\frac{252-2}{2}\right\rfloor+1126 x 126 x 32000
3 conv\left\lfloor\frac{126+2 \times 0-5}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{126+2 \times 0-5}{1}\right\rfloor+1122 x 122 x 6451,2006451,264
4 pool\left\lfloor\frac{122-2}{2}\right\rfloor+1LaTeX: \left\lfloor\frac{122-2}{2}\right\rfloor+161 x 61 x 64000
5 conv\left\lfloor\frac{61+2 \times 1-5}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{61+2 \times 1-5}{1}\right\rfloor+159 x 59 x 128204,800128204,928
6 pool\left\lfloor\frac{59-2}{2}\right\rfloor+1LaTeX: \left\lfloor\frac{59-2}{2}\right\rfloor+129 x 29 x 128000
7 flat29 × 29 × 128107648000
8 dense107648 → 84849,042,432849,042,516
9 dense84 → 32322,688322,720
10 dense32 → 1132133

Parameter substitutions, in layer order:

Layer 1: (5 x 5 x 3 + 1) x 32 = 2,432.

Layer 3: (5 x 5 x 32 + 1) x 64 = 51,264.

Layer 5: (5 x 5 x 64 + 1) x 128 = 204,928.

Layer 8: (107648 + 1) x 84 = 9,042,516.

Layer 9: (84 + 1) x 32 = 2,720.

Layer 10: (32 + 1) x 1 = 33.

Total: 2,432 + 51,264 + 204,928 + 9,042,516 + 2,720 + 33 = 9,303,893 trainable parameters. Pooling changes activation size; it does not itself add learned weights. Always floor noninteger spatial dimensions.

Correction: Conv 3 has (5 × 5 × 64+1) × 128 = 204,928 parameters, not 73,856 as printed in the key. Correct total is 9,303,893. If binary classification is instead implemented with two softmax outputs, the last dense layer has(32+1) × 2 = 66 parameters and the total becomes 9,303,926; explicitly state the output convention.