Deep Learning · MT2 · section

Pooling needs comparisons or averages

Convolutional neural networks

Definition. Pooling is a local aggregation operation that summarizes each spatial window, usually independently per channel. Max pooling selects the largest value; average pooling calculates a mean. Standard pooling has no learned weights or biases.

Explanation and working

Max pooling selects the largest value inside each window. Average pooling adds window entries and divides by their count. Ordinary max and average pooling have no learned weights or biases. Pool each input channel independently, so the number of channels is unchanged.

Question. Pool this 4 × 4 map with 2 × 2 windows, stride 2 and no padding:

Given

The 4 × 4 input map is printed below.

Window = 2 × 2; stride = 2; padding = 0.

To find

Find the pooled output by the operation(s) requested.

Formula — which one and why

Pooling. Max pooling selects the largest window entry. Average pooling adds its entries and divides by their count. Each channel is pooled separately.

V = current pooling window; n = number of entries in that window. Pooling learns no weights or biases.

$y_{\max}=\max V$ LaTeX: y_{\max}=\max V

$y_{\mathrm{avg}}=\frac1n\sum_{v\in V}v$ LaTeX: y_{\mathrm{avg}}=\frac1n\sum_{v\in V}v

Yaad rakhne ka cue: Max mein biggest; average mein total ÷ window ka count.

Solution — step by step

$A=\begin{bmatrix}21&11&15&10\\12&9&10&11\\11&13&18&12\\7&4&15&12\end{bmatrix}$ LaTeX: A=\begin{bmatrix}21&11&15&10\\12&9&10&11\\11&13&18&12\\7&4&15&12\end{bmatrix}

Top-left window

Step 1: select the four entries.

[21, 11, 12, 9]

Step 2: choose the largest entry.

$y_{\max}=21$ LaTeX: y_{\max}=21

Step 3: add all entries.

$21+11+12+9=53$ LaTeX: 21+11+12+9=53

Step 4: divide the sum by four for the average.

$y_{\mathrm{avg}}=\frac{53}4=13.25$ LaTeX: y_{\mathrm{avg}}=\frac{53}4=13.25

Top-right window

Step 1: select the four entries.

[15, 10, 10, 11]

Step 2: choose the largest entry.

$y_{\max}=15$ LaTeX: y_{\max}=15

Step 3: add all entries.

$15+10+10+11=46$ LaTeX: 15+10+10+11=46

Step 4: divide the sum by four for the average.

$y_{\mathrm{avg}}=\frac{46}4=11.5$ LaTeX: y_{\mathrm{avg}}=\frac{46}4=11.5

Bottom-left window

Step 1: select the four entries.

[11, 13, 7, 4]

Step 2: choose the largest entry.

$y_{\max}=13$ LaTeX: y_{\max}=13

Step 3: add all entries.

$11+13+7+4=35$ LaTeX: 11+13+7+4=35

Step 4: divide the sum by four for the average.

$y_{\mathrm{avg}}=\frac{35}4=8.75$ LaTeX: y_{\mathrm{avg}}=\frac{35}4=8.75

Bottom-right window

Step 1: select the four entries.

[18, 12, 15, 12]

Step 2: choose the largest entry.

$y_{\max}=18$ LaTeX: y_{\max}=18

Step 3: add all entries.

$18+12+15+12=57$ LaTeX: 18+12+15+12=57

Step 4: divide the sum by four for the average.

$y_{\mathrm{avg}}=\frac{57}4=14.25$ LaTeX: y_{\mathrm{avg}}=\frac{57}4=14.25

Outputs are max $\begin{bmatrix}21 & 15\\13 & 18\end{bmatrix}$ and average $\begin{bmatrix}13.25 & 11.5\\8.75 & 14.25\end{bmatrix}$ .

Each pooling window reduces several values to one Tap the diagram to enlarge.

Pooling reduces spatial size and later computation and can provide limited robustness to small shifts. It can lose exact position information. It does not guarantee complete translation invariance.