Deep Learning · TEE · formulas

Convolution output dimensions

This formula counts how many legal filter positions fit along one image dimension. Padding adds space at both ends, the kernel occupies K positions, and stride controls how far the next window moves. The floor keeps only complete valid positions.

N_{out}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{out}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

Reading image shapes before doing CNN arithmetic

Definition. An image or intermediate CNN representation is a tensor indexed by spatial position and channel. This course writes its shape as height × width × channels, excluding the batch dimension.

Explanation and working

Height, width and channels

The shape 4 × 5 × 3 means 4 rows, 5 columns and 3 channels. It contains 4 × 5 × 3 = 60 numbers. A grayscale image commonly has one intensity channel; an RGB image has red, green and blue channels. Intermediate CNN channels are learned feature maps, not necessarily colors.

Count height, width and channels separately. Three channels are stacked at the same spatial positions. Tap the diagram to enlarge.

One position versus the whole map

A filter placed at one location computes one output value. Sliding that same filter over all valid locations creates one feature map. A second filter creates a second feature map. Therefore the number of filters equals the number of output channels.

Why filter depth matches input channels

A 3 × 3 filter applied to a 3-channel input has 3 × 3 × 3 = 27 weights. At one location, it combines 27 input values into one sum. A single bias is then added for that filter. It does not produce three separate output channels just because the input has three channels.

Reading a complete shape calculation

The spatial-size formula computes height and width. It does not compute the channel count.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

N = current input size; P = padding on one side; K = filter size; S = stride. The floor symbol means keep only complete placements. Add one because the initial placement is also an output position.

Shape, parameters and computation are three different counts

Shape counts the arrangement of output values. Parameters count learned weights and biases. Computation counts how many arithmetic operations are performed. Reusing the same filter at 100 positions creates 100 outputs but does not create 100 new sets of weights.

In a multi-layer question, write the output shape of a layer first. That becomes the next layer's input shape. This is why the second convolution uses the first convolution's channel count, not the original image's channel count.

Stride padding and floor

Definition. Stride is the spacing between consecutive kernel positions. Padding extends the input boundary. Together with kernel size and dilation, they determine the spatial dimensions of a convolutional output.

Explanation and working

Stride is how far the filter moves each time. Stride 2 visits starting positions 0, 2, 4 and so on. Padding adds border values, usually zeros. Padding 1 means one new row or column on each side, adding 2 to a spatial dimension.

The border adds to both sides of the input. The stride changes the next starting position of the filter window. Tap the diagram to enlarge.
N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

N is input size; P is padding per side; K is kernel size; S is stride. Floor means take the greatest integer not above the result: \left\lfloor 2.5\right\rfloorLaTeX: \left\lfloor 2.5\right\rfloor = 2. It counts only positions where the entire window fits. The formula assumes dilation 1, the standard setting in these papers. Apply it separately to height and width for rectangular images or kernels.

Why plus one? Even when the kernel exactly fits the image, there is one placement. If N = K, P = 0, S = 1, the formula is 0 + 1 = 1.

Output dimension with a fractional intermediate result

Question. A convolution has input side N = 6, kernel side K = 3, stride S = 2 and padding P = 0. Find its output side.

Given

N = 6; K = 3; S = 2; P = 0; dilation = 1.

To find

The integer output side length.

Formula — which one and why

The formula counts complete kernel placements. Flooring removes an incomplete final placement.

N is input side; P is padding on each side; K is kernel side; S is stride.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

Yaad rakhne ka cue: Padding dono side; stride se divide; floor; phir +1.

Solution — step by step

Step 1: include padding and subtract kernel size.

6+2(0)-3=3LaTeX: 6+2(0)-3=3

Step 2: divide by the stride.

\frac32=1.5LaTeX: \frac32=1.5

Step 3: take the floor.

\lfloor1.5\rfloor=1LaTeX: \lfloor1.5\rfloor=1

Step 4: count the first placement.

N_{\mathrm{out}}=1+1=2LaTeX: N_{\mathrm{out}}=1+1=2

Final answer: Output side = 2; a square input gives a 2 × 2 map per filter.

Padding on both sides

Question. A square convolution input has side 5, kernel side 3, stride 1 and padding 1 on each side. Find the padded-input size and output size.

Given

N = 5; K = 3; S = 1; P = 1 on each side.

To find

Padded input and output dimensions.

Formula — which one and why

Padding adds P entries on each of two opposite sides; the output counts complete kernel placements.

N = unpadded side; K = kernel side; S = stride; P = per-side padding.

N_{\mathrm{padded}}=N+2PLaTeX: N_{\mathrm{padded}}=N+2P
N_{\mathrm{out}}=\lfloor(N+2P-K)/S\rfloor+1LaTeX: N_{\mathrm{out}}=\lfloor(N+2P-K)/S\rfloor+1

Yaad rakhne ka cue: Padded input aur output ka size alag cheez hain.

Solution — step by step

Step 1: count both padding edges.

2P=2(1)=2LaTeX: 2P=2(1)=2

Step 2: calculate the padded input side.

N+2P=5+2=7LaTeX: N+2P=5+2=7

Step 3: subtract kernel width.

7-3=4LaTeX: 7-3=4

Step 4: divide by stride and take the floor.

\lfloor4/1\rfloor=4LaTeX: \lfloor4/1\rfloor=4

Step 5: include the first placement.

N_{out}=4+1=5LaTeX: N_{out}=4+1=5

Final answer: Padded input = 7 × 7; output = 5 × 5 per filter.

“Valid” usually means P = 0. For odd K and S = 1, padding (K − 1)/2 preserves the spatial size. For S greater than 1, software “same” padding can be asymmetric and commonly gives ceiling(N/S); do not force the stride-1 rule into that case.

Even-sized kernels: SAME padding is not always symmetric

Definition. A numerical padding value specifies how many entries to add at each boundary. SAME is an output-size convention. At stride 1, SAME preserves height and width; an even-sized kernel can require unequal padding on opposite sides.

For a 2 by 2 kernel at stride 1, preserving the input size requires one padding entry in total per axis, not one on each side. For example, add no row above and one row below, and no column on the left and one on the right. Reversing which side receives the extra entry preserves the output size too, but changes which input cells contribute at each output location; follow the question's convention.

Hinglish cue: P = 1 means dono taraf ek; SAME means output ka size bachao. Dono instructions ko same mat samjho.

Example 1: one entry on every side

Question. A 5 × 5 single-channel input is processed by one 2 × 2 convolution filter. The stride is 1. Exactly one zero is padded on each of the four sides. Find the padded-input size and output size. (Practice: 3 marks)

Given

Input height and width N = 5; kernel size K = 2; stride S = 1; padding P = 1 on each side; dilation = 1; one output filter.

To find

The padded input dimensions and the convolution output dimensions.

Formula — which one and why

The two opposite borders contribute P entries each. The output formula counts complete filter placements after padding.

N_{\mathrm{padded}}=N+2PLaTeX: N_{\mathrm{padded}}=N+2P
N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

N is one input dimension; P is padding on each side; K is kernel size; S is stride. Apply the same calculation separately to height and width. One filter produces one output channel.

Solution — step by step

Step 1: count the two padding borders.

2P=2(1)=2LaTeX: 2P=2(1)=2

Step 2: find the padded side length.

N_{\mathrm{padded}}=5+2=7LaTeX: N_{\mathrm{padded}}=5+2=7

The padded input is 7 × 7 × 1.

Step 3: subtract the kernel size.

7-2=5LaTeX: 7-2=5

Step 4: divide by stride, take the floor and count the first placement.

N_{\mathrm{out}}=\left\lfloor\frac{5}{1}\right\rfloor+1=5+1=6LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{5}{1}\right\rfloor+1=5+1=6

Step 5: write height, width and channels.

Both spatial dimensions equal 6. There is one filter, so there is one output channel.

Final answer: padded input = 7 × 7 × 1; output = 6 × 6 × 1. This explicitly specified padding does not implement SAME for this even kernel.

Example 2: preserve the size with SAME padding

Question. The same 5 × 5 single-channel input is processed by one 2 × 2 convolution filter with stride 1 and SAME padding. Find the total padding required along each axis and the output size. Use zero padding above and to the left, with any required extra padding below and to the right. (Practice: 3 marks)

Given

Input side N = 5; kernel side K = 2; stride S = 1; desired output side Nout = 5; dilation = 1; one filter. Padding before the input is 0; padding after the input must be found.

To find

Padding above, below, left and right; padded-input shape; final output shape.

Formula — which one and why

For unequal padding, replace 2P by the sum of the before and after padding. Pbefore and Pafter are the entries added on the two opposite sides of one axis.

N_{\mathrm{out}}=\left\lfloor\frac{N+P_{\mathrm{before}}+P_{\mathrm{after}}-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+P_{\mathrm{before}}+P_{\mathrm{after}}-K}{S}\right\rfloor+1

At stride 1, the quantities are integers. To keep Nout = N, the total padding must equal K − 1.

P_{\mathrm{total}}=P_{\mathrm{before}}+P_{\mathrm{after}}=K-1LaTeX: P_{\mathrm{total}}=P_{\mathrm{before}}+P_{\mathrm{after}}=K-1

Solution — step by step

Step 1: calculate total padding along the height.

P_{\mathrm{total}}=2-1=1LaTeX: P_{\mathrm{total}}=2-1=1

Only one extra row in total is needed.

Step 2: place the row using the stated convention.

Padding above = 0; padding below = 1.

Step 3: apply the same calculation to width.

Only one extra column in total is needed. Padding left = 0; padding right = 1.

Step 4: find the padded-input shape.

H_{\mathrm{padded}}=5+0+1=6LaTeX: H_{\mathrm{padded}}=5+0+1=6
W_{\mathrm{padded}}=5+0+1=6LaTeX: W_{\mathrm{padded}}=5+0+1=6

The padded input is 6 × 6 × 1.

Step 5: count the output positions along each axis.

N_{\mathrm{out}}=\left\lfloor\frac{5+0+1-2}{1}\right\rfloor+1=4+1=5LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{5+0+1-2}{1}\right\rfloor+1=4+1=5

Step 6: include the output channel.

One filter creates one output channel.

Final answer: add 0 above, 1 below, 0 left and 1 right; padded input = 6 × 6 × 1; output = 5 × 5 × 1.

The two questions have the same input and kernel but different padding instructions. Read the stated boundary padding before applying the formula.

Drill 35 Padding stride and channels

Question

Question. Input 31 × 31 × 3; 5 × 5 convolution,8 filters, stride 2, padding 1. Find shape, weights, biases, total.

Answer — step by step

Given

Input shape = 31 × 31 × 3.

Filter size = 5 × 5; number of filters = 8.

Stride S = 2; padding P = 1 per side; one bias per filter.

To find

Find output shape, weights, biases and total parameters.

Formula — which one and why

Convolution/pooling spatial size. Padding adds P positions at both ends. Subtract kernel width to find the last valid start, divide by stride, take the floor and include the initial placement with +1.

N = current height or width; K = kernel/window size; S = stride; P = padding per side. Apply separately to height and width; dilation is 1 here.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

Yaad rakhne ka cue: Input + dono sides ki padding − kernel; stride se divide; floor + 1.

Convolution parameters. Every filter spans all input channels. Count its spatial coefficients across those channels, add one bias and multiply by the number of filters.

K_h,K_w = filter height/width; C_in = input channels; C_out = number of filters and output channels. Spatial output positions reuse these weights.

N_w=K_hK_wC_{\mathrm{in}}C_{\mathrm{out}}LaTeX: N_w=K_hK_wC_{\mathrm{in}}C_{\mathrm{out}}
N_b=C_{\mathrm{out}}LaTeX: N_b=C_{\mathrm{out}}
P=N_w+N_bLaTeX: P=N_w+N_b

Yaad rakhne ka cue: Filter area × input channels × filters; phir ek bias per filter.

Solution — step by step

Meaning of each given value

The input has height 31, width 31 and 3 channels. K = 5 is the filter width/height; S = 2 is the stride; P = 1 is padding on each side. Eight filters create eight output channels.

Formulas used

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1
\text{weights}=K^2C_{\mathrm{in}}C_{\mathrm{out}},\quad \text{biases}=C_{\mathrm{out}}LaTeX: \text{weights}=K^2C_{\mathrm{in}}C_{\mathrm{out}},\quad \text{biases}=C_{\mathrm{out}}
\text{parameters}=\text{weights}+\text{biases}LaTeX: \text{parameters}=\text{weights}+\text{biases}

Step 1: find the output height and width

Padding adds one cell on each side, so the padded width is 31 + 2 × 1 = 33.

N_{\mathrm{out}}=\left\lfloor\frac{31+2(1)-5}{2}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{31+2(1)-5}{2}\right\rfloor+1
N_{\mathrm{out}}=\left\lfloor\frac{28}{2}\right\rfloor+1=14+1=15LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{28}{2}\right\rfloor+1=14+1=15

The height uses the same calculation. Output shape = 15 × 15 × 8.

Step 2: count weights in one filter

Each filter covers 5 × 5 positions in each of 3 channels.

5\times5\times3=75\text{ weights per filter}LaTeX: 5\times5\times3=75\text{ weights per filter}

Step 3: count all filters and their biases

\text{weights}=75\times8=600LaTeX: \text{weights}=75\times8=600
\text{biases}=8,\qquad \text{total}=600+8=608LaTeX: \text{biases}=8,\qquad \text{total}=600+8=608

Final answer: shape 15 × 15 × 8; 600 weights; 8 biases; 608 parameters. The 15 × 15 output locations reuse those weights.