# F22 Q3b Subject: Deep Learning Exam: MT2 Reader: https://deep-learning-study-room.vercel.app/#mt2/read/15/mt2-c15-s4 Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c15-s4/ ## Question **Question (10 marks).** Count trainable parameters of the illustrated LeNet-style digit classifier with convolution 5 × 5: 6 filters, pooling 2; convolution 5 × 5: 16 filters, pooling 2; dense 120, dense 84, softmax 10. Use the consistent 32 × 32 × 1 input that produces the drawn 28,14,10,5 spatial maps. ## Answer — step by step ### Given Count trainable parameters of the illustrated LeNet-style digit classifier with convolution 5 × 5: 6 filters, pooling 2. convolution 5 × 5: 16 filters, pooling 2. dense 120, dense 84, softmax 10. Use the consistent 32 × 32 × 1 input that produces the drawn 28,14,10,5 spatial maps. ### To find Find every layer shape, its parameter count and the network total. ### Formula — which one and why **Complete CNN calculation.** Carry each output shape into the next layer. Convolution changes spatial size and channel count; pooling changes spatial size; flattening converts all activations into dense-layer inputs. H,W,C = feature-map height, width and channels; K = kernel size; S = stride; P\_pad = padding. One bias is assumed for each learned filter or dense neuron unless specified otherwise. $$ N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1 $$ $$ P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}} $$ $$ n_{\mathrm{flat}}=HWC $$ $$ P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}} $$ $$ P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0 $$ **Yaad rakhne ka cue:** Ek layer ka output, next layer ka input; shape aur parameters alag count. ### Solution — step by step Use output size $\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1$ , Conv weights K² × Cin × Cout and biases Cout, Dense weights Nin × Nout and biases Nout. Assume a bias per learned filter/unit, full input-channel connectivity and no trainable pooling parameters. ReLU and softmax activations themselves add no trainable parameters. ### Layer-by-layer calculation Carry the complete output shape of one layer into the next. Spatial size determines the number of activations; it does not multiply the number of shared convolution parameters. ### Layer 1 — convolution Layer 1: what changes and what is learned Input shape**32 × 32 × 1**height × width × channels → Convolution**6 filters · 5 × 5 · S = 1 · P = 0** → Output shape**28 × 28 × 6**height × width × channels Weights**150** + Biases**6** = Trainable parameters**156** Spatial positions share the filter weights. Output depth equals the number of filters. Input: **32 × 32 × 1**. Use 6 filters of spatial size 5 × 5, stride 1, and padding 0 on each side. **1. Calculate height and width.** $$ N_{\mathrm{out}}=\left\lfloor\frac{32+2(0)-5}{1}\right\rfloor+1=28 $$ **2. Determine output channels.** Each filter produces one feature map. Therefore the output is **28 × 28 × 6**. **3. Count trainable values.** Each filter receives all 1 input channels. Its weights are reused at every spatial position. $$ N_{\mathrm{weights}}=5\times5\times1\times6=150 $$ $$ N_{\mathrm{biases}}=6 $$ $$ P_1=150+6=156 $$ ### Layer 2 — pooling Layer 2: what changes and what is learned Input shape**28 × 28 × 6**height × width × channels → Pooling**2 × 2 window · S = 2** → Output shape**14 × 14 × 6**height × width × channels Weights**0** + Biases**0** = Trainable parameters**0** Spatial dimensions change; the channel count stays the same. Pooling learns no weights. Input: **28 × 28 × 6**. Use a 2 × 2 window, stride 2, and zero padding. $$ N_{\mathrm{out}}=\left\lfloor\frac{28-2}{2}\right\rfloor+1=14 $$ Pooling acts independently on each channel. The output is **14 × 14 × 6**. **Trainable parameters: 0.** Max/average pooling has no learned filter weights or biases. ### Layer 3 — convolution Layer 3: what changes and what is learned Input shape**14 × 14 × 6**height × width × channels → Convolution**16 filters · 5 × 5 · S = 1 · P = 0** → Output shape**10 × 10 × 16**height × width × channels Weights**2,400** + Biases**16** = Trainable parameters**2,416** Spatial positions share the filter weights. Output depth equals the number of filters. Input: **14 × 14 × 6**. Use 16 filters of spatial size 5 × 5, stride 1, and padding 0 on each side. **1. Calculate height and width.** $$ N_{\mathrm{out}}=\left\lfloor\frac{14+2(0)-5}{1}\right\rfloor+1=10 $$ **2. Determine output channels.** Each filter produces one feature map. Therefore the output is **10 × 10 × 16**. **3. Count trainable values.** Each filter receives all 6 input channels. Its weights are reused at every spatial position. $$ N_{\mathrm{weights}}=5\times5\times6\times16=2{,}400 $$ $$ N_{\mathrm{biases}}=16 $$ $$ P_3=2{,}400+16=2{,}416 $$ ### Layer 4 — pooling Layer 4: what changes and what is learned Input shape**10 × 10 × 16**height × width × channels → Pooling**2 × 2 window · S = 2** → Output shape**5 × 5 × 16**height × width × channels Weights**0** + Biases**0** = Trainable parameters**0** Spatial dimensions change; the channel count stays the same. Pooling learns no weights. Input: **10 × 10 × 16**. Use a 2 × 2 window, stride 2, and zero padding. $$ N_{\mathrm{out}}=\left\lfloor\frac{10-2}{2}\right\rfloor+1=5 $$ Pooling acts independently on each channel. The output is **5 × 5 × 16**. **Trainable parameters: 0.** Max/average pooling has no learned filter weights or biases. ### Layer 5 — flatten Layer 5: what changes and what is learned Input shape**5 × 5 × 16**height × width × channels → Flatten**Rearrange activations** → Output shape**400**output features Weights**0** + Biases**0** = Trainable parameters**0** The same activation values are rearranged into one vector. No parameter is learned. Input: **5 × 5 × 16**. Rearrange all activation values into one vector. $$ n_{\mathrm{flat}}=5\times5\times16=400 $$ Output: a vector with **400 entries**. Flattening changes the arrangement, so it adds **0 trainable parameters**. ### Layer 6 — fully connected layer Layer 6: what changes and what is learned Input shape**400**input features → Dense layer**120 neurons** → Output shape**120**output features Weights**48,000** + Biases**120** = Trainable parameters**48,120** Every output neuron receives every input feature, with one bias per output neuron. Input: **400 features**. Output: **120 neurons**. Every output neuron connects to every input feature and has one bias. $$ N_{\mathrm{weights}}=400\times120=48{,}000 $$ $$ N_{\mathrm{biases}}=120 $$ $$ P_6=48{,}000+120=48{,}120 $$ ### Layer 7 — fully connected layer Layer 7: what changes and what is learned Input shape**120**input features → Dense layer**84 neurons** → Output shape**84**output features Weights**10,080** + Biases**84** = Trainable parameters**10,164** Every output neuron receives every input feature, with one bias per output neuron. Input: **120 features**. Output: **84 neurons**. Every output neuron connects to every input feature and has one bias. $$ N_{\mathrm{weights}}=120\times84=10{,}080 $$ $$ N_{\mathrm{biases}}=84 $$ $$ P_7=10{,}080+84=10{,}164 $$ ### Layer 8 — fully connected layer Layer 8: what changes and what is learned Input shape**84**input features → Dense layer**10 neurons** → Output shape**10**output features Weights**840** + Biases**10** = Trainable parameters**850** Every output neuron receives every input feature, with one bias per output neuron. Input: **84 features**. Output: **10 neurons**. Every output neuron connects to every input feature and has one bias. $$ N_{\mathrm{weights}}=84\times10=840 $$ $$ N_{\mathrm{biases}}=10 $$ $$ P_8=840+10=850 $$ ### Summary of the complete network | Layer | Spatial or flatten calculation | Output | Weights | Biases | Total | | --- | --- | --- | --- | --- | --- | | 1 conv | $\left\lfloor\frac{32+2 \times 0-5}{1}\right\rfloor+1$ | 28 x 28 x 6 | 150 | 6 | 156 | | 2 pool | $\left\lfloor\frac{28-2}{2}\right\rfloor+1$ | 14 x 14 x 6 | 0 | 0 | 0 | | 3 conv | $\left\lfloor\frac{14+2 \times 0-5}{1}\right\rfloor+1$ | 10 x 10 x 16 | 2,400 | 16 | 2,416 | | 4 pool | $\left\lfloor\frac{10-2}{2}\right\rfloor+1$ | 5 x 5 x 16 | 0 | 0 | 0 | | 5 flat | 5 × 5 × 16 | 400 | 0 | 0 | 0 | | 6 dense | 400 → 120 | 120 | 48,000 | 120 | 48,120 | | 7 dense | 120 → 84 | 84 | 10,080 | 84 | 10,164 | | 8 dense | 84 → 10 | 10 | 840 | 10 | 850 | **Parameter substitutions, in layer order:** Layer 1: (5 x 5 x 1 + 1) x 6 = 156. Layer 3: (5 x 5 x 6 + 1) x 16 = 2,416. Layer 6: (400 + 1) x 120 = 48,120. Layer 7: (120 + 1) x 84 = 10,164. Layer 8: (84 + 1) x 10 = 850. **Total:** 156 + 2,416 + 48,120 + 10,164 + 850 = **61,706 trainable parameters**. Pooling changes activation size; it does not itself add learned weights. Always floor noninteger spatial dimensions. This is the standard classroom full-channel interpretation with nontrainable pooling and a dense 10-class softmax. The historical original LeNet used partial channel connections and other details, so its published parameter count need not equal this diagram-based count. If an input label is read as 28, the stated later 28 map after valid 5 × 5 convolution is inconsistent; record the 32 input assumption rather than silently ignoring the mismatch.