Deep Learning · TEE · section

R24 Q4b

Re examination February 2024 fully solved

Question

Question (10 marks). Calculate weights, biases and feature-map sizes of the CNN: Input 227 × 227 × 3. → Conv K = 9, filters = 32, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Conv K = 5, filters = 64, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Conv K = 5, filters = 64, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Flatten. → Dense 120 units. → Dense 84 units. → Dense 10 units.

Answer — step by step

Given

Input 227 × 227 × 3.

Conv K = 9, filters = 32, stride = 1, padding = 0.

Pool K = 2, stride = 2, padding = 0.

Conv K = 5, filters = 64, stride = 1, padding = 0.

Pool K = 2, stride = 2, padding = 0.

Conv K = 5, filters = 64, stride = 1, padding = 0.

Pool K = 2, stride = 2, padding = 0.

Dense 120 units.

Dense 84 units.

Dense 10 units.

To find

Find every layer shape, its parameter count and the network total.

Formula — which one and why

Complete CNN calculation. Carry each output shape into the next layer. Convolution changes spatial size and channel count; pooling changes spatial size; flattening converts all activations into dense-layer inputs.

H,W,C = feature-map height, width and channels; K = kernel size; S = stride; P_pad = padding. One bias is assumed for each learned filter or dense neuron unless specified otherwise.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1
P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}LaTeX: P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}
n_{\mathrm{flat}}=HWCLaTeX: n_{\mathrm{flat}}=HWC
P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}LaTeX: P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}
P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0LaTeX: P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0

Yaad rakhne ka cue: Ek layer ka output, next layer ka input; shape aur parameters alag count.

Solution — step by step

Use output size \left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: \left\lfloor\frac{N+2P-K}{S}\right\rfloor+1, Conv weights K² × Cin × Cout and biases Cout, Dense weights Nin × Nout and biases Nout. Assume a bias per learned filter/unit, full input-channel connectivity and no trainable pooling parameters. ReLU and softmax activations themselves add no trainable parameters.

Layer-by-layer calculation

Carry the complete output shape of one layer into the next. Spatial size determines the number of activations; it does not multiply the number of shared convolution parameters.

Layer 1 — convolution

Layer 1: what changes and what is learned
Input shape227 × 227 × 3height × width × channels
Convolution32 filters · 9 × 9 · S = 1 · P = 0
Output shape219 × 219 × 32height × width × channels
Weights7,776
Biases32
Trainable parameters7,808

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 227 × 227 × 3. Use 32 filters of spatial size 9 × 9, stride 1, and padding 0 on each side.

1. Calculate height and width.

N_{\mathrm{out}}=\left\lfloor\frac{227+2(0)-9}{1}\right\rfloor+1=219LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{227+2(0)-9}{1}\right\rfloor+1=219

2. Determine output channels. Each filter produces one feature map. Therefore the output is 219 × 219 × 32.

3. Count trainable values. Each filter receives all 3 input channels. Its weights are reused at every spatial position.

N_{\mathrm{weights}}=9\times9\times3\times32=7{,}776LaTeX: N_{\mathrm{weights}}=9\times9\times3\times32=7{,}776
N_{\mathrm{biases}}=32LaTeX: N_{\mathrm{biases}}=32
P_1=7{,}776+32=7{,}808LaTeX: P_1=7{,}776+32=7{,}808

Layer 2 — pooling

Layer 2: what changes and what is learned
Input shape219 × 219 × 32height × width × channels
Pooling2 × 2 window · S = 2
Output shape109 × 109 × 32height × width × channels
Weights0
Biases0
Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 219 × 219 × 32. Use a 2 × 2 window, stride 2, and zero padding.

N_{\mathrm{out}}=\left\lfloor\frac{219-2}{2}\right\rfloor+1=109LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{219-2}{2}\right\rfloor+1=109

Pooling acts independently on each channel. The output is 109 × 109 × 32.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

The division gives 108.5; take its floor before adding 1. A partial window outside the unpadded input is not included.

Layer 3 — convolution

Layer 3: what changes and what is learned
Input shape109 × 109 × 32height × width × channels
Convolution64 filters · 5 × 5 · S = 1 · P = 0
Output shape105 × 105 × 64height × width × channels
Weights51,200
Biases64
Trainable parameters51,264

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 109 × 109 × 32. Use 64 filters of spatial size 5 × 5, stride 1, and padding 0 on each side.

1. Calculate height and width.

N_{\mathrm{out}}=\left\lfloor\frac{109+2(0)-5}{1}\right\rfloor+1=105LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{109+2(0)-5}{1}\right\rfloor+1=105

2. Determine output channels. Each filter produces one feature map. Therefore the output is 105 × 105 × 64.

3. Count trainable values. Each filter receives all 32 input channels. Its weights are reused at every spatial position.

N_{\mathrm{weights}}=5\times5\times32\times64=51{,}200LaTeX: N_{\mathrm{weights}}=5\times5\times32\times64=51{,}200
N_{\mathrm{biases}}=64LaTeX: N_{\mathrm{biases}}=64
P_3=51{,}200+64=51{,}264LaTeX: P_3=51{,}200+64=51{,}264

Layer 4 — pooling

Layer 4: what changes and what is learned
Input shape105 × 105 × 64height × width × channels
Pooling2 × 2 window · S = 2
Output shape52 × 52 × 64height × width × channels
Weights0
Biases0
Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 105 × 105 × 64. Use a 2 × 2 window, stride 2, and zero padding.

N_{\mathrm{out}}=\left\lfloor\frac{105-2}{2}\right\rfloor+1=52LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{105-2}{2}\right\rfloor+1=52

Pooling acts independently on each channel. The output is 52 × 52 × 64.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

The division gives 51.5; take its floor before adding 1. A partial window outside the unpadded input is not included.

Layer 5 — convolution

Layer 5: what changes and what is learned
Input shape52 × 52 × 64height × width × channels
Convolution64 filters · 5 × 5 · S = 1 · P = 0
Output shape48 × 48 × 64height × width × channels
Weights102,400
Biases64
Trainable parameters102,464

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 52 × 52 × 64. Use 64 filters of spatial size 5 × 5, stride 1, and padding 0 on each side.

1. Calculate height and width.

N_{\mathrm{out}}=\left\lfloor\frac{52+2(0)-5}{1}\right\rfloor+1=48LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{52+2(0)-5}{1}\right\rfloor+1=48

2. Determine output channels. Each filter produces one feature map. Therefore the output is 48 × 48 × 64.

3. Count trainable values. Each filter receives all 64 input channels. Its weights are reused at every spatial position.

N_{\mathrm{weights}}=5\times5\times64\times64=102{,}400LaTeX: N_{\mathrm{weights}}=5\times5\times64\times64=102{,}400
N_{\mathrm{biases}}=64LaTeX: N_{\mathrm{biases}}=64
P_5=102{,}400+64=102{,}464LaTeX: P_5=102{,}400+64=102{,}464

Layer 6 — pooling

Layer 6: what changes and what is learned
Input shape48 × 48 × 64height × width × channels
Pooling2 × 2 window · S = 2
Output shape24 × 24 × 64height × width × channels
Weights0
Biases0
Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 48 × 48 × 64. Use a 2 × 2 window, stride 2, and zero padding.

N_{\mathrm{out}}=\left\lfloor\frac{48-2}{2}\right\rfloor+1=24LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{48-2}{2}\right\rfloor+1=24

Pooling acts independently on each channel. The output is 24 × 24 × 64.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

Layer 7 — flatten

Layer 7: what changes and what is learned
Input shape24 × 24 × 64height × width × channels
FlattenRearrange activations
Output shape36,864output features
Weights0
Biases0
Trainable parameters0

The same activation values are rearranged into one vector. No parameter is learned.

Input: 24 × 24 × 64. Rearrange all activation values into one vector.

n_{\mathrm{flat}}=24\times24\times64=36{,}864LaTeX: n_{\mathrm{flat}}=24\times24\times64=36{,}864

Output: a vector with 36,864 entries. Flattening changes the arrangement, so it adds 0 trainable parameters.

Layer 8 — fully connected layer

Layer 8: what changes and what is learned
Input shape36,864input features
Dense layer120 neurons
Output shape120output features
Weights4,423,680
Biases120
Trainable parameters4,423,800

Every output neuron receives every input feature, with one bias per output neuron.

Input: 36,864 features. Output: 120 neurons. Every output neuron connects to every input feature and has one bias.

N_{\mathrm{weights}}=36{,}864\times120=4{,}423{,}680LaTeX: N_{\mathrm{weights}}=36{,}864\times120=4{,}423{,}680
N_{\mathrm{biases}}=120LaTeX: N_{\mathrm{biases}}=120
P_8=4{,}423{,}680+120=4{,}423{,}800LaTeX: P_8=4{,}423{,}680+120=4{,}423{,}800

Layer 9 — fully connected layer

Layer 9: what changes and what is learned
Input shape120input features
Dense layer84 neurons
Output shape84output features
Weights10,080
Biases84
Trainable parameters10,164

Every output neuron receives every input feature, with one bias per output neuron.

Input: 120 features. Output: 84 neurons. Every output neuron connects to every input feature and has one bias.

N_{\mathrm{weights}}=120\times84=10{,}080LaTeX: N_{\mathrm{weights}}=120\times84=10{,}080
N_{\mathrm{biases}}=84LaTeX: N_{\mathrm{biases}}=84
P_9=10{,}080+84=10{,}164LaTeX: P_9=10{,}080+84=10{,}164

Layer 10 — fully connected layer

Layer 10: what changes and what is learned
Input shape84input features
Dense layer10 neurons
Output shape10output features
Weights840
Biases10
Trainable parameters850

Every output neuron receives every input feature, with one bias per output neuron.

Input: 84 features. Output: 10 neurons. Every output neuron connects to every input feature and has one bias.

N_{\mathrm{weights}}=84\times10=840LaTeX: N_{\mathrm{weights}}=84\times10=840
N_{\mathrm{biases}}=10LaTeX: N_{\mathrm{biases}}=10
P_10=840+10=850LaTeX: P_10=840+10=850

Summary of the complete network

LayerSpatial or flatten calculationOutputWeightsBiasesTotal
1 conv\left\lfloor\frac{227+2 \times 0-9}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{227+2 \times 0-9}{1}\right\rfloor+1219 x 219 x 327,776327,808
2 pool\left\lfloor\frac{219-2}{2}\right\rfloor+1LaTeX: \left\lfloor\frac{219-2}{2}\right\rfloor+1109 x 109 x 32000
3 conv\left\lfloor\frac{109+2 \times 0-5}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{109+2 \times 0-5}{1}\right\rfloor+1105 x 105 x 6451,2006451,264
4 pool\left\lfloor\frac{105-2}{2}\right\rfloor+1LaTeX: \left\lfloor\frac{105-2}{2}\right\rfloor+152 x 52 x 64000
5 conv\left\lfloor\frac{52+2 \times 0-5}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{52+2 \times 0-5}{1}\right\rfloor+148 x 48 x 64102,40064102,464
6 pool\left\lfloor\frac{48-2}{2}\right\rfloor+1LaTeX: \left\lfloor\frac{48-2}{2}\right\rfloor+124 x 24 x 64000
7 flat24 × 24 × 6436864000
8 dense36864 → 1201204,423,6801204,423,800
9 dense120 → 848410,0808410,164
10 dense84 → 101084010850

Parameter substitutions, in layer order:

Layer 1: (9 x 9 x 3 + 1) x 32 = 7,808.

Layer 3: (5 x 5 x 32 + 1) x 64 = 51,264.

Layer 5: (5 x 5 x 64 + 1) x 64 = 102,464.

Layer 8: (36864 + 1) x 120 = 4,423,800.

Layer 9: (120 + 1) x 84 = 10,164.

Layer 10: (84 + 1) x 10 = 850.

Total: 7,808 + 51,264 + 102,464 + 4,423,800 + 10,164 + 850 = 4,596,350 trainable parameters. Pooling changes activation size; it does not itself add learned weights. Always floor noninteger spatial dimensions.