# R24 Q4b Subject: Deep Learning Exam: MT2 Reader: https://deep-learning-study-room.vercel.app/#mt2/read/14/mt2-c14-s4 Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c14-s4/ ## Question **Question (10 marks).** Calculate weights, biases and feature-map sizes of the CNN: Input 227 × 227 × 3. → Conv K = 9, filters = 32, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Conv K = 5, filters = 64, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Conv K = 5, filters = 64, stride = 1, padding = 0. → Pool K = 2, stride = 2, padding = 0. → Flatten. → Dense 120 units. → Dense 84 units. → Dense 10 units. ## Answer — step by step ### Given Input 227 × 227 × 3. Conv K = 9, filters = 32, stride = 1, padding = 0. Pool K = 2, stride = 2, padding = 0. Conv K = 5, filters = 64, stride = 1, padding = 0. Pool K = 2, stride = 2, padding = 0. Conv K = 5, filters = 64, stride = 1, padding = 0. Pool K = 2, stride = 2, padding = 0. Dense 120 units. Dense 84 units. Dense 10 units. ### To find Find every layer shape, its parameter count and the network total. ### Formula — which one and why **Complete CNN calculation.** Carry each output shape into the next layer. Convolution changes spatial size and channel count; pooling changes spatial size; flattening converts all activations into dense-layer inputs. H,W,C = feature-map height, width and channels; K = kernel size; S = stride; P\_pad = padding. One bias is assumed for each learned filter or dense neuron unless specified otherwise. $$ N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1 $$ $$ P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}} $$ $$ n_{\mathrm{flat}}=HWC $$ $$ P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}} $$ $$ P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0 $$ **Yaad rakhne ka cue:** Ek layer ka output, next layer ka input; shape aur parameters alag count. ### Solution — step by step Use output size $\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1$ , Conv weights K² × Cin × Cout and biases Cout, Dense weights Nin × Nout and biases Nout. Assume a bias per learned filter/unit, full input-channel connectivity and no trainable pooling parameters. ReLU and softmax activations themselves add no trainable parameters. ### Layer-by-layer calculation Carry the complete output shape of one layer into the next. Spatial size determines the number of activations; it does not multiply the number of shared convolution parameters. ### Layer 1 — convolution Layer 1: what changes and what is learned Input shape**227 × 227 × 3**height × width × channels → Convolution**32 filters · 9 × 9 · S = 1 · P = 0** → Output shape**219 × 219 × 32**height × width × channels Weights**7,776** + Biases**32** = Trainable parameters**7,808** Spatial positions share the filter weights. Output depth equals the number of filters. Input: **227 × 227 × 3**. Use 32 filters of spatial size 9 × 9, stride 1, and padding 0 on each side. **1. Calculate height and width.** $$ N_{\mathrm{out}}=\left\lfloor\frac{227+2(0)-9}{1}\right\rfloor+1=219 $$ **2. Determine output channels.** Each filter produces one feature map. Therefore the output is **219 × 219 × 32**. **3. Count trainable values.** Each filter receives all 3 input channels. Its weights are reused at every spatial position. $$ N_{\mathrm{weights}}=9\times9\times3\times32=7{,}776 $$ $$ N_{\mathrm{biases}}=32 $$ $$ P_1=7{,}776+32=7{,}808 $$ ### Layer 2 — pooling Layer 2: what changes and what is learned Input shape**219 × 219 × 32**height × width × channels → Pooling**2 × 2 window · S = 2** → Output shape**109 × 109 × 32**height × width × channels Weights**0** + Biases**0** = Trainable parameters**0** Spatial dimensions change; the channel count stays the same. Pooling learns no weights. Input: **219 × 219 × 32**. Use a 2 × 2 window, stride 2, and zero padding. $$ N_{\mathrm{out}}=\left\lfloor\frac{219-2}{2}\right\rfloor+1=109 $$ Pooling acts independently on each channel. The output is **109 × 109 × 32**. **Trainable parameters: 0.** Max/average pooling has no learned filter weights or biases. The division gives 108.5; take its floor before adding 1. A partial window outside the unpadded input is not included. ### Layer 3 — convolution Layer 3: what changes and what is learned Input shape**109 × 109 × 32**height × width × channels → Convolution**64 filters · 5 × 5 · S = 1 · P = 0** → Output shape**105 × 105 × 64**height × width × channels Weights**51,200** + Biases**64** = Trainable parameters**51,264** Spatial positions share the filter weights. Output depth equals the number of filters. Input: **109 × 109 × 32**. Use 64 filters of spatial size 5 × 5, stride 1, and padding 0 on each side. **1. Calculate height and width.** $$ N_{\mathrm{out}}=\left\lfloor\frac{109+2(0)-5}{1}\right\rfloor+1=105 $$ **2. Determine output channels.** Each filter produces one feature map. Therefore the output is **105 × 105 × 64**. **3. Count trainable values.** Each filter receives all 32 input channels. Its weights are reused at every spatial position. $$ N_{\mathrm{weights}}=5\times5\times32\times64=51{,}200 $$ $$ N_{\mathrm{biases}}=64 $$ $$ P_3=51{,}200+64=51{,}264 $$ ### Layer 4 — pooling Layer 4: what changes and what is learned Input shape**105 × 105 × 64**height × width × channels → Pooling**2 × 2 window · S = 2** → Output shape**52 × 52 × 64**height × width × channels Weights**0** + Biases**0** = Trainable parameters**0** Spatial dimensions change; the channel count stays the same. Pooling learns no weights. Input: **105 × 105 × 64**. Use a 2 × 2 window, stride 2, and zero padding. $$ N_{\mathrm{out}}=\left\lfloor\frac{105-2}{2}\right\rfloor+1=52 $$ Pooling acts independently on each channel. The output is **52 × 52 × 64**. **Trainable parameters: 0.** Max/average pooling has no learned filter weights or biases. The division gives 51.5; take its floor before adding 1. A partial window outside the unpadded input is not included. ### Layer 5 — convolution Layer 5: what changes and what is learned Input shape**52 × 52 × 64**height × width × channels → Convolution**64 filters · 5 × 5 · S = 1 · P = 0** → Output shape**48 × 48 × 64**height × width × channels Weights**102,400** + Biases**64** = Trainable parameters**102,464** Spatial positions share the filter weights. Output depth equals the number of filters. Input: **52 × 52 × 64**. Use 64 filters of spatial size 5 × 5, stride 1, and padding 0 on each side. **1. Calculate height and width.** $$ N_{\mathrm{out}}=\left\lfloor\frac{52+2(0)-5}{1}\right\rfloor+1=48 $$ **2. Determine output channels.** Each filter produces one feature map. Therefore the output is **48 × 48 × 64**. **3. Count trainable values.** Each filter receives all 64 input channels. Its weights are reused at every spatial position. $$ N_{\mathrm{weights}}=5\times5\times64\times64=102{,}400 $$ $$ N_{\mathrm{biases}}=64 $$ $$ P_5=102{,}400+64=102{,}464 $$ ### Layer 6 — pooling Layer 6: what changes and what is learned Input shape**48 × 48 × 64**height × width × channels → Pooling**2 × 2 window · S = 2** → Output shape**24 × 24 × 64**height × width × channels Weights**0** + Biases**0** = Trainable parameters**0** Spatial dimensions change; the channel count stays the same. Pooling learns no weights. Input: **48 × 48 × 64**. Use a 2 × 2 window, stride 2, and zero padding. $$ N_{\mathrm{out}}=\left\lfloor\frac{48-2}{2}\right\rfloor+1=24 $$ Pooling acts independently on each channel. The output is **24 × 24 × 64**. **Trainable parameters: 0.** Max/average pooling has no learned filter weights or biases. ### Layer 7 — flatten Layer 7: what changes and what is learned Input shape**24 × 24 × 64**height × width × channels → Flatten**Rearrange activations** → Output shape**36,864**output features Weights**0** + Biases**0** = Trainable parameters**0** The same activation values are rearranged into one vector. No parameter is learned. Input: **24 × 24 × 64**. Rearrange all activation values into one vector. $$ n_{\mathrm{flat}}=24\times24\times64=36{,}864 $$ Output: a vector with **36,864 entries**. Flattening changes the arrangement, so it adds **0 trainable parameters**. ### Layer 8 — fully connected layer Layer 8: what changes and what is learned Input shape**36,864**input features → Dense layer**120 neurons** → Output shape**120**output features Weights**4,423,680** + Biases**120** = Trainable parameters**4,423,800** Every output neuron receives every input feature, with one bias per output neuron. Input: **36,864 features**. Output: **120 neurons**. Every output neuron connects to every input feature and has one bias. $$ N_{\mathrm{weights}}=36{,}864\times120=4{,}423{,}680 $$ $$ N_{\mathrm{biases}}=120 $$ $$ P_8=4{,}423{,}680+120=4{,}423{,}800 $$ ### Layer 9 — fully connected layer Layer 9: what changes and what is learned Input shape**120**input features → Dense layer**84 neurons** → Output shape**84**output features Weights**10,080** + Biases**84** = Trainable parameters**10,164** Every output neuron receives every input feature, with one bias per output neuron. Input: **120 features**. Output: **84 neurons**. Every output neuron connects to every input feature and has one bias. $$ N_{\mathrm{weights}}=120\times84=10{,}080 $$ $$ N_{\mathrm{biases}}=84 $$ $$ P_9=10{,}080+84=10{,}164 $$ ### Layer 10 — fully connected layer Layer 10: what changes and what is learned Input shape**84**input features → Dense layer**10 neurons** → Output shape**10**output features Weights**840** + Biases**10** = Trainable parameters**850** Every output neuron receives every input feature, with one bias per output neuron. Input: **84 features**. Output: **10 neurons**. Every output neuron connects to every input feature and has one bias. $$ N_{\mathrm{weights}}=84\times10=840 $$ $$ N_{\mathrm{biases}}=10 $$ $$ P_10=840+10=850 $$ ### Summary of the complete network | Layer | Spatial or flatten calculation | Output | Weights | Biases | Total | | --- | --- | --- | --- | --- | --- | | 1 conv | $\left\lfloor\frac{227+2 \times 0-9}{1}\right\rfloor+1$ | 219 x 219 x 32 | 7,776 | 32 | 7,808 | | 2 pool | $\left\lfloor\frac{219-2}{2}\right\rfloor+1$ | 109 x 109 x 32 | 0 | 0 | 0 | | 3 conv | $\left\lfloor\frac{109+2 \times 0-5}{1}\right\rfloor+1$ | 105 x 105 x 64 | 51,200 | 64 | 51,264 | | 4 pool | $\left\lfloor\frac{105-2}{2}\right\rfloor+1$ | 52 x 52 x 64 | 0 | 0 | 0 | | 5 conv | $\left\lfloor\frac{52+2 \times 0-5}{1}\right\rfloor+1$ | 48 x 48 x 64 | 102,400 | 64 | 102,464 | | 6 pool | $\left\lfloor\frac{48-2}{2}\right\rfloor+1$ | 24 x 24 x 64 | 0 | 0 | 0 | | 7 flat | 24 × 24 × 64 | 36864 | 0 | 0 | 0 | | 8 dense | 36864 → 120 | 120 | 4,423,680 | 120 | 4,423,800 | | 9 dense | 120 → 84 | 84 | 10,080 | 84 | 10,164 | | 10 dense | 84 → 10 | 10 | 840 | 10 | 850 | **Parameter substitutions, in layer order:** Layer 1: (9 x 9 x 3 + 1) x 32 = 7,808. Layer 3: (5 x 5 x 32 + 1) x 64 = 51,264. Layer 5: (5 x 5 x 64 + 1) x 64 = 102,464. Layer 8: (36864 + 1) x 120 = 4,423,800. Layer 9: (120 + 1) x 84 = 10,164. Layer 10: (84 + 1) x 10 = 850. **Total:** 7,808 + 51,264 + 102,464 + 4,423,800 + 10,164 + 850 = **4,596,350 trainable parameters**. Pooling changes activation size; it does not itself add learned weights. Always floor noninteger spatial dimensions.