Deep Learning · TEE · section

Drill 37 Complete small CNN

Numerical practice from direct to mixed

Question

Question. Input 10 × 10 × 1 → Conv 3 × 3,4 filters, stride 1, valid → Pool 2 stride 2 → Flatten → Dense 3. Find all shapes and parameters.

Answer — step by step

Given

Input = 10 × 10 × 1.

Conv: 3 × 3, 4 filters, stride 1, valid padding.

Pool: 2 × 2, stride 2; then flatten and dense layer with 3 outputs.

Include one bias per filter/dense neuron.

To find

Find all output shapes, layer parameter counts and their sum.

Formula — which one and why

Complete CNN calculation. Carry each output shape into the next layer. Convolution changes spatial size and channel count; pooling changes spatial size; flattening converts all activations into dense-layer inputs.

H,W,C = feature-map height, width and channels; K = kernel size; S = stride; P_pad = padding. One bias is assumed for each learned filter or dense neuron unless specified otherwise.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1
P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}LaTeX: P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}
n_{\mathrm{flat}}=HWCLaTeX: n_{\mathrm{flat}}=HWC
P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}LaTeX: P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}
P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0LaTeX: P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0

Yaad rakhne ka cue: Ek layer ka output, next layer ka input; shape aur parameters alag count.

Solution — step by step

Formulas and assumptions

“Valid” means zero padding. The convolution and dense layers each have one bias per receiving filter or neuron. Pooling and flattening have no learned parameters.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1
P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}LaTeX: P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}
n_{\mathrm{flat}}=HWC,\qquad P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}LaTeX: n_{\mathrm{flat}}=HWC,\qquad P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}

Step 1: convolution shape

N_{\mathrm{conv}}=\frac{10+0-3}{1}+1=7+1=8LaTeX: N_{\mathrm{conv}}=\frac{10+0-3}{1}+1=7+1=8

Four filters produce four channels: 8 × 8 × 4.

Step 2: convolution parameters

\text{weights}=3\times3\times1\times4=36LaTeX: \text{weights}=3\times3\times1\times4=36
\text{biases}=4,\qquad P_{\mathrm{conv}}=36+4=40LaTeX: \text{biases}=4,\qquad P_{\mathrm{conv}}=36+4=40

Step 3: pooling shape

Pooling now receives size 8, not the original size 10.

N_{\mathrm{pool}}=\left\lfloor\frac{8-2}{2}\right\rfloor+1=3+1=4LaTeX: N_{\mathrm{pool}}=\left\lfloor\frac{8-2}{2}\right\rfloor+1=3+1=4

Channels remain 4: 4 × 4 × 4. Pooling parameters = 0.

Step 4: flatten

n_{\mathrm{flat}}=4\times4\times4=64LaTeX: n_{\mathrm{flat}}=4\times4\times4=64

Flatten creates a vector of 64 numbers. It rearranges values, so its parameter count is 0.

Step 5: dense layer with 3 outputs

\text{weights}=64\times3=192LaTeX: \text{weights}=64\times3=192
\text{biases}=3,\qquad P_{\mathrm{dense}}=192+3=195LaTeX: \text{biases}=3,\qquad P_{\mathrm{dense}}=192+3=195

Step 6: add learned parameters

P_{\mathrm{total}}=40+0+0+195=235LaTeX: P_{\mathrm{total}}=40+0+0+195=235

Final answer: shapes 8 × 8 × 4 → 4 × 4 × 4 → 64 → 3; total 235 parameters.