Deep Learning · MT2 · section

Max, min and average pooling on the same lecture grid

Convolutional neural networks

Definition. Pooling summarizes each local window separately within each channel. Max pooling keeps its largest value, min pooling its smallest value, and average pooling its arithmetic mean. Ordinary versions have no trainable weights or biases.

Question. Apply 2 × 2 max, min and average pooling with stride 2 and no padding to the following 4 × 4 input. Give all three output matrices. (Practice: 3 marks)

Given

X = $\begin{bmatrix}12 & 20 & 30 & 0\\8 & 12 & 2 & 0\\34 & 70 & 37 & 4\\112 & 100 & 25 & 12\end{bmatrix}$ . Window = 2 × 2; stride = 2; padding = 0.

To find

The shared output shape and each of the four cells for each pooling method.

Formula — which one and why

Each output summarizes four input numbers. Use max for the largest, min for the smallest, and sum divided by 4 for the average. The window moves two cells, so these four windows do not overlap.

$y_{\max}=\max(x_1,x_2,x_3,x_4)$ LaTeX: y_{\max}=\max(x_1,x_2,x_3,x_4)

$y_{\min}=\min(x_1,x_2,x_3,x_4)$ LaTeX: y_{\min}=\min(x_1,x_2,x_3,x_4)

$y_{\mathrm{avg}}=\frac{x_1+x_2+x_3+x_4}{4}$ LaTeX: y_{\mathrm{avg}}=\frac{x_1+x_2+x_3+x_4}{4}

The four colours identify four separate pooling windows. Each window creates one cell in each output map. Tap the diagram to enlarge.

Solution — step by step

Step 1: output size. (4 −2)/2 +1 = 2. Each output has shape 2 × 2.

Step 2: top-left window [12,20,8,12]. Largest = 20; smallest = 8. Sum = 12+20+8+12 = 52. Average = 52/4 = 13.

Step 3: top-right window [30,0,2,0]. Largest = 30; smallest = 0. Sum = 30+0+2+0 = 32. Average = 32/4 = 8.

Step 4: bottom-left window [34,70,112,100]. Largest = 112; smallest = 34. Sum = 34+70+112+100 = 316. Average = 316/4 = 79.

Step 5: bottom-right window [37,4,25,12]. Largest = 37; smallest = 4. Sum = 37+4+25+12 = 78. Average = 78/4 = 19.5. Do not silently round this to 20.

Step 6: put each result back in its window's position. Max = $\begin{bmatrix}20 & 30\\112 & 37\end{bmatrix}$ . Min = $\begin{bmatrix}8 & 0\\34 & 4\end{bmatrix}$ . Average = $\begin{bmatrix}13 & 8\\79 & 19.5\end{bmatrix}$ .

Final answer: Three 2 × 2 maps as above; zero trainable parameters for each pooling operation.

Why the choice matters

Max retains the largest activation in a feature window, which can still be negative if every entry is negative. Min retains the lowest response. Average retains an overall response but can dilute a sharp local peak. On raw intensity images, min can retain a dark stroke and max can retain a bright stroke. A learned CNN feature map represents filter responses; the best pooling choice is not determined solely by the original background colour.

Pooling has no parameters to update, but gradients can still pass through it. For a unique max, the upstream gradient is routed to the winning input. For an average over four inputs, each receives one-fourth of that gradient. Fewer pooled activations can reduce a later dense layer's parameter count; they do not change the size of an earlier convolution's filter.

Yaad rakho: Max = sabse bada; min = sabse chhota; average = sab jodo, count se divide.