Deep Learning · TEE · section

F22 Q3b

Final examination November 2022 fully solved

Question

Question (10 marks). Count trainable parameters of the illustrated LeNet-style digit classifier with convolution 5 × 5: 6 filters, pooling 2; convolution 5 × 5: 16 filters, pooling 2; dense 120, dense 84, softmax 10. Use the consistent 32 × 32 × 1 input that produces the drawn 28,14,10,5 spatial maps.

Answer — step by step

Given

Count trainable parameters of the illustrated LeNet-style digit classifier with convolution 5 × 5: 6 filters, pooling 2.

convolution 5 × 5: 16 filters, pooling 2.

dense 120, dense 84, softmax 10.

Use the consistent 32 × 32 × 1 input that produces the drawn 28,14,10,5 spatial maps.

To find

Find every layer shape, its parameter count and the network total.

Formula — which one and why

Complete CNN calculation. Carry each output shape into the next layer. Convolution changes spatial size and channel count; pooling changes spatial size; flattening converts all activations into dense-layer inputs.

H,W,C = feature-map height, width and channels; K = kernel size; S = stride; P_pad = padding. One bias is assumed for each learned filter or dense neuron unless specified otherwise.

$P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}$ LaTeX: P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}

$n_{\mathrm{flat}}=HWC$ LaTeX: n_{\mathrm{flat}}=HWC

$P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}$ LaTeX: P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}

$P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0$ LaTeX: P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0

Yaad rakhne ka cue: Ek layer ka output, next layer ka input; shape aur parameters alag count.

Solution — step by step

Use output size $\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1$ , Conv weights K² × Cin × Cout and biases Cout, Dense weights Nin × Nout and biases Nout. Assume a bias per learned filter/unit, full input-channel connectivity and no trainable pooling parameters. ReLU and softmax activations themselves add no trainable parameters.

Layer-by-layer calculation

Carry the complete output shape of one layer into the next. Spatial size determines the number of activations; it does not multiply the number of shared convolution parameters.

Layer 1 — convolution

Layer 1: what changes and what is learned

Input shape32 × 32 × 1height × width × channels

Convolution6 filters · 5 × 5 · S = 1 · P = 0

Output shape28 × 28 × 6height × width × channels

Weights150

Biases6

Trainable parameters156

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 32 × 32 × 1. Use 6 filters of spatial size 5 × 5, stride 1, and padding 0 on each side.

1. Calculate height and width.

2. Determine output channels. Each filter produces one feature map. Therefore the output is 28 × 28 × 6.

3. Count trainable values. Each filter receives all 1 input channels. Its weights are reused at every spatial position.

$N_{\mathrm{biases}}=6$ LaTeX: N_{\mathrm{biases}}=6

$P_1=150+6=156$ LaTeX: P_1=150+6=156

Layer 2 — pooling

Layer 2: what changes and what is learned

Input shape28 × 28 × 6height × width × channels

Pooling2 × 2 window · S = 2

Output shape14 × 14 × 6height × width × channels

Weights0

Biases0

Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 28 × 28 × 6. Use a 2 × 2 window, stride 2, and zero padding.

$N_{\mathrm{out}}=\left\lfloor\frac{28-2}{2}\right\rfloor+1=14$ LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{28-2}{2}\right\rfloor+1=14

Pooling acts independently on each channel. The output is 14 × 14 × 6.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

Layer 3 — convolution

Layer 3: what changes and what is learned

Input shape14 × 14 × 6height × width × channels

Convolution16 filters · 5 × 5 · S = 1 · P = 0

Output shape10 × 10 × 16height × width × channels

Weights2,400

Biases16

Trainable parameters2,416

Spatial positions share the filter weights. Output depth equals the number of filters.

Input: 14 × 14 × 6. Use 16 filters of spatial size 5 × 5, stride 1, and padding 0 on each side.

1. Calculate height and width.

2. Determine output channels. Each filter produces one feature map. Therefore the output is 10 × 10 × 16.

3. Count trainable values. Each filter receives all 6 input channels. Its weights are reused at every spatial position.

$N_{\mathrm{biases}}=16$ LaTeX: N_{\mathrm{biases}}=16

$P_3=2{,}400+16=2{,}416$ LaTeX: P_3=2{,}400+16=2{,}416

Layer 4 — pooling

Layer 4: what changes and what is learned

Input shape10 × 10 × 16height × width × channels

Pooling2 × 2 window · S = 2

Output shape5 × 5 × 16height × width × channels

Weights0

Biases0

Trainable parameters0

Spatial dimensions change; the channel count stays the same. Pooling learns no weights.

Input: 10 × 10 × 16. Use a 2 × 2 window, stride 2, and zero padding.

$N_{\mathrm{out}}=\left\lfloor\frac{10-2}{2}\right\rfloor+1=5$ LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{10-2}{2}\right\rfloor+1=5

Pooling acts independently on each channel. The output is 5 × 5 × 16.

Trainable parameters: 0. Max/average pooling has no learned filter weights or biases.

Layer 5 — flatten

Layer 5: what changes and what is learned

Input shape5 × 5 × 16height × width × channels

FlattenRearrange activations

Output shape400output features

Weights0

Biases0

Trainable parameters0

The same activation values are rearranged into one vector. No parameter is learned.

Input: 5 × 5 × 16. Rearrange all activation values into one vector.

$n_{\mathrm{flat}}=5\times5\times16=400$ LaTeX: n_{\mathrm{flat}}=5\times5\times16=400

Output: a vector with 400 entries. Flattening changes the arrangement, so it adds 0 trainable parameters.

Layer 6 — fully connected layer

Layer 6: what changes and what is learned

Input shape400input features

Dense layer120 neurons

Output shape120output features

Weights48,000

Biases120

Trainable parameters48,120

Every output neuron receives every input feature, with one bias per output neuron.

Input: 400 features. Output: 120 neurons. Every output neuron connects to every input feature and has one bias.

$N_{\mathrm{biases}}=120$ LaTeX: N_{\mathrm{biases}}=120

Layer 7 — fully connected layer

Layer 7: what changes and what is learned

Input shape120input features

Dense layer84 neurons

Output shape84output features

Weights10,080

Biases84

Trainable parameters10,164

Every output neuron receives every input feature, with one bias per output neuron.

Input: 120 features. Output: 84 neurons. Every output neuron connects to every input feature and has one bias.

$N_{\mathrm{biases}}=84$ LaTeX: N_{\mathrm{biases}}=84

$P_7=10{,}080+84=10{,}164$ LaTeX: P_7=10{,}080+84=10{,}164

Layer 8 — fully connected layer

Layer 8: what changes and what is learned

Input shape84input features

Dense layer10 neurons

Output shape10output features

Weights840

Biases10

Trainable parameters850

Every output neuron receives every input feature, with one bias per output neuron.

Input: 84 features. Output: 10 neurons. Every output neuron connects to every input feature and has one bias.

$N_{\mathrm{weights}}=84\times10=840$ LaTeX: N_{\mathrm{weights}}=84\times10=840

$N_{\mathrm{biases}}=10$ LaTeX: N_{\mathrm{biases}}=10

$P_8=840+10=850$ LaTeX: P_8=840+10=850

Summary of the complete network

Layer	Spatial or flatten calculation	Output	Weights	Biases	Total
1 conv	$\left\lfloor\frac{32+2 \times 0-5}{1}\right\rfloor+1$	28 x 28 x 6	150	6	156
2 pool	$\left\lfloor\frac{28-2}{2}\right\rfloor+1$	14 x 14 x 6	0	0	0
3 conv	$\left\lfloor\frac{14+2 \times 0-5}{1}\right\rfloor+1$	10 x 10 x 16	2,400	16	2,416
4 pool	$\left\lfloor\frac{10-2}{2}\right\rfloor+1$	5 x 5 x 16	0	0	0
5 flat	5 × 5 × 16	400	0	0	0
6 dense	400 → 120	120	48,000	120	48,120
7 dense	120 → 84	84	10,080	84	10,164
8 dense	84 → 10	10	840	10	850

Parameter substitutions, in layer order:

Layer 1: (5 x 5 x 1 + 1) x 6 = 156.

Layer 3: (5 x 5 x 6 + 1) x 16 = 2,416.

Layer 6: (400 + 1) x 120 = 48,120.

Layer 7: (120 + 1) x 84 = 10,164.

Layer 8: (84 + 1) x 10 = 850.

Total: 156 + 2,416 + 48,120 + 10,164 + 850 = 61,706 trainable parameters. Pooling changes activation size; it does not itself add learned weights. Always floor noninteger spatial dimensions.

This is the standard classroom full-channel interpretation with nontrainable pooling and a dense 10-class softmax. The historical original LeNet used partial channel connections and other details, so its published parameter count need not equal this diagram-based count. If an input label is read as 28, the stated later 28 map after valid 5 × 5 convolution is inconsistent; record the 32 input assumption rather than silently ignoring the mismatch.