Deep Learning · TEE · section

Drill 52 Projection shortcut

Numerical practice from direct to mixed

Question

Question. A residual main path outputs 8 × 8 × 16 but input is 16 × 16 × 8. Design a 1 × 1 projection shortcut with stride 2, zero padding. Find its output and parameters.

Answer — step by step

Given

Residual input = 16 × 16 × 8; main-path output = 8 × 8 × 16.

Shortcut kernel = 1 × 1; stride 2; padding 0.

Use one bias per projection filter.

To find

Choose the number of filters and find projection output shape and parameters.

Formula — which one and why

Convolution/pooling spatial size. Padding adds P positions at both ends. Subtract kernel width to find the last valid start, divide by stride, take the floor and include the initial placement with +1.

N = current height or width; K = kernel/window size; S = stride; P = padding per side. Apply separately to height and width; dilation is 1 here.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

Yaad rakhne ka cue: Input + dono sides ki padding − kernel; stride se divide; floor + 1.

Convolution parameters. Every filter spans all input channels. Count its spatial coefficients across those channels, add one bias and multiply by the number of filters.

K_h,K_w = filter height/width; C_in = input channels; C_out = number of filters and output channels. Spatial output positions reuse these weights.

N_w=K_hK_wC_{\mathrm{in}}C_{\mathrm{out}}LaTeX: N_w=K_hK_wC_{\mathrm{in}}C_{\mathrm{out}}
N_b=C_{\mathrm{out}}LaTeX: N_b=C_{\mathrm{out}}
P=N_w+N_bLaTeX: P=N_w+N_b

Yaad rakhne ka cue: Filter area × input channels × filters; phir ek bias per filter.

Solution — step by step

Meaning and formulas

Residual addition needs identical shapes on both paths. The projection must reduce height and width from 16 to 8 and increase channels from 8 to 16.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1
P_{\mathrm{projection}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}LaTeX: P_{\mathrm{projection}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}

Step 1: choose 16 filters

One filter creates one channel, so 16 filters give the required 16 output channels.

Step 2: use the specified 1 × 1 kernel and stride 2

N_{\mathrm{out}}=\left\lfloor\frac{16+0-1}{2}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{16+0-1}{2}\right\rfloor+1
N_{\mathrm{out}}=\lfloor7.5\rfloor+1=7+1=8LaTeX: N_{\mathrm{out}}=\lfloor7.5\rfloor+1=7+1=8

The floor operation drops the fractional part; it does not round 7.5 to 8 before adding 1.

Step 3: count weights and biases

\text{weights}=1\times1\times8\times16=128LaTeX: \text{weights}=1\times1\times8\times16=128
\text{biases}=16,\qquad \text{total}=128+16=144LaTeX: \text{biases}=16,\qquad \text{total}=128+16=144

Final answer: projection output 8 × 8 × 16 and 144 parameters with biases. The projected shortcut now matches the main path, so elementwise addition is valid.