Deep Learning · TEE · formulas

Max, min and average pooling

Pooling reduces each window to one value. Max pooling keeps the largest entry; average pooling adds all included entries and divides by their count. Pooling is performed independently in each channel.

y_{\min}=\min(x_1,x_2,x_3,x_4)LaTeX: y_{\min}=\min(x_1,x_2,x_3,x_4)
y_{max}=\max_{(i,j)\in W}x_{ij}LaTeX: y_{max}=\max_{(i,j)\in W}x_{ij}
y_{avg}=\frac1{|W|}\sum_{(i,j)\in W}x_{ij}LaTeX: y_{avg}=\frac1{|W|}\sum_{(i,j)\in W}x_{ij}

Pooling needs comparisons or averages

Definition. Pooling is a local aggregation operation that summarizes each spatial window, usually independently per channel. Max pooling selects the largest value; average pooling calculates a mean. Standard pooling has no learned weights or biases.

Explanation and working

Max pooling selects the largest value inside each window. Average pooling adds window entries and divides by their count. Ordinary max and average pooling have no learned weights or biases. Pool each input channel independently, so the number of channels is unchanged.

Question. Pool this 4 × 4 map with 2 × 2 windows, stride 2 and no padding:

Given

The 4 × 4 input map is printed below.

Window = 2 × 2; stride = 2; padding = 0.

To find

Find the pooled output by the operation(s) requested.

Formula — which one and why

Pooling. Max pooling selects the largest window entry. Average pooling adds its entries and divides by their count. Each channel is pooled separately.

V = current pooling window; n = number of entries in that window. Pooling learns no weights or biases.

y_{\max}=\max VLaTeX: y_{\max}=\max V
y_{\mathrm{avg}}=\frac1n\sum_{v\in V}vLaTeX: y_{\mathrm{avg}}=\frac1n\sum_{v\in V}v

Yaad rakhne ka cue: Max mein biggest; average mein total ÷ window ka count.

Solution — step by step

A=\begin{bmatrix}21&11&15&10\\12&9&10&11\\11&13&18&12\\7&4&15&12\end{bmatrix}LaTeX: A=\begin{bmatrix}21&11&15&10\\12&9&10&11\\11&13&18&12\\7&4&15&12\end{bmatrix}

Top-left window

Step 1: select the four entries.

[21, 11, 12, 9]

Step 2: choose the largest entry.

y_{\max}=21LaTeX: y_{\max}=21

Step 3: add all entries.

21+11+12+9=53LaTeX: 21+11+12+9=53

Step 4: divide the sum by four for the average.

y_{\mathrm{avg}}=\frac{53}4=13.25LaTeX: y_{\mathrm{avg}}=\frac{53}4=13.25

Top-right window

Step 1: select the four entries.

[15, 10, 10, 11]

Step 2: choose the largest entry.

y_{\max}=15LaTeX: y_{\max}=15

Step 3: add all entries.

15+10+10+11=46LaTeX: 15+10+10+11=46

Step 4: divide the sum by four for the average.

y_{\mathrm{avg}}=\frac{46}4=11.5LaTeX: y_{\mathrm{avg}}=\frac{46}4=11.5

Bottom-left window

Step 1: select the four entries.

[11, 13, 7, 4]

Step 2: choose the largest entry.

y_{\max}=13LaTeX: y_{\max}=13

Step 3: add all entries.

11+13+7+4=35LaTeX: 11+13+7+4=35

Step 4: divide the sum by four for the average.

y_{\mathrm{avg}}=\frac{35}4=8.75LaTeX: y_{\mathrm{avg}}=\frac{35}4=8.75

Bottom-right window

Step 1: select the four entries.

[18, 12, 15, 12]

Step 2: choose the largest entry.

y_{\max}=18LaTeX: y_{\max}=18

Step 3: add all entries.

18+12+15+12=57LaTeX: 18+12+15+12=57

Step 4: divide the sum by four for the average.

y_{\mathrm{avg}}=\frac{57}4=14.25LaTeX: y_{\mathrm{avg}}=\frac{57}4=14.25

Outputs are max \begin{bmatrix}21 & 15\\13 & 18\end{bmatrix}LaTeX: \begin{bmatrix}21 & 15\\13 & 18\end{bmatrix} and average \begin{bmatrix}13.25 & 11.5\\8.75 & 14.25\end{bmatrix}LaTeX: \begin{bmatrix}13.25 & 11.5\\8.75 & 14.25\end{bmatrix}.

Each pooling window reduces several values to one Tap the diagram to enlarge.

Pooling reduces spatial size and later computation and can provide limited robustness to small shifts. It can lose exact position information. It does not guarantee complete translation invariance.

Max, min and average pooling on the same lecture grid

Definition. Pooling summarizes each local window separately within each channel. Max pooling keeps its largest value, min pooling its smallest value, and average pooling its arithmetic mean. Ordinary versions have no trainable weights or biases.

Question. Apply 2 × 2 max, min and average pooling with stride 2 and no padding to the following 4 × 4 input. Give all three output matrices. (Practice: 3 marks)

Given

X = \begin{bmatrix}12 & 20 & 30 & 0\\8 & 12 & 2 & 0\\34 & 70 & 37 & 4\\112 & 100 & 25 & 12\end{bmatrix}LaTeX: \begin{bmatrix}12 & 20 & 30 & 0\\8 & 12 & 2 & 0\\34 & 70 & 37 & 4\\112 & 100 & 25 & 12\end{bmatrix}. Window = 2 × 2; stride = 2; padding = 0.

To find

The shared output shape and each of the four cells for each pooling method.

Formula — which one and why

Each output summarizes four input numbers. Use max for the largest, min for the smallest, and sum divided by 4 for the average. The window moves two cells, so these four windows do not overlap.

y_{\max}=\max(x_1,x_2,x_3,x_4)LaTeX: y_{\max}=\max(x_1,x_2,x_3,x_4)
y_{\min}=\min(x_1,x_2,x_3,x_4)LaTeX: y_{\min}=\min(x_1,x_2,x_3,x_4)
y_{\mathrm{avg}}=\frac{x_1+x_2+x_3+x_4}{4}LaTeX: y_{\mathrm{avg}}=\frac{x_1+x_2+x_3+x_4}{4}
The four colours identify four separate pooling windows. Each window creates one cell in each output map. Tap the diagram to enlarge.

Solution — step by step

Step 1: output size. (4 −2)/2 +1 = 2. Each output has shape 2 × 2.

Step 2: top-left window [12,20,8,12]. Largest = 20; smallest = 8. Sum = 12+20+8+12 = 52. Average = 52/4 = 13.

Step 3: top-right window [30,0,2,0]. Largest = 30; smallest = 0. Sum = 30+0+2+0 = 32. Average = 32/4 = 8.

Step 4: bottom-left window [34,70,112,100]. Largest = 112; smallest = 34. Sum = 34+70+112+100 = 316. Average = 316/4 = 79.

Step 5: bottom-right window [37,4,25,12]. Largest = 37; smallest = 4. Sum = 37+4+25+12 = 78. Average = 78/4 = 19.5. Do not silently round this to 20.

Step 6: put each result back in its window's position. Max = \begin{bmatrix}20 & 30\\112 & 37\end{bmatrix}LaTeX: \begin{bmatrix}20 & 30\\112 & 37\end{bmatrix}. Min = \begin{bmatrix}8 & 0\\34 & 4\end{bmatrix}LaTeX: \begin{bmatrix}8 & 0\\34 & 4\end{bmatrix}. Average = \begin{bmatrix}13 & 8\\79 & 19.5\end{bmatrix}LaTeX: \begin{bmatrix}13 & 8\\79 & 19.5\end{bmatrix}.

Final answer: Three 2 × 2 maps as above; zero trainable parameters for each pooling operation.

Why the choice matters

Max retains the largest activation in a feature window, which can still be negative if every entry is negative. Min retains the lowest response. Average retains an overall response but can dilute a sharp local peak. On raw intensity images, min can retain a dark stroke and max can retain a bright stroke. A learned CNN feature map represents filter responses; the best pooling choice is not determined solely by the original background colour.

Pooling has no parameters to update, but gradients can still pass through it. For a unique max, the upstream gradient is routed to the winning input. For an average over four inputs, each receives one-fourth of that gradient. Fewer pooled activations can reduce a later dense layer's parameter count; they do not change the size of an earlier convolution's filter.

Yaad rakho: Max = sabse bada; min = sabse chhota; average = sab jodo, count se divide.

Drill 36 Max and average pooling

Question

Question. Pool the input A shown below with 2 × 2 windows and stride 2. Find max and average outputs.

Given matrices for this question Tap the diagram to enlarge.

Answer — step by step

Given

Input A is printed with the question.

Window size = 2 × 2; stride = 2; no padding.

To find

Find the complete max-pooling and average-pooling outputs.

Formula — which one and why

Pooling. Max pooling selects the largest window entry. Average pooling adds its entries and divides by their count. Each channel is pooled separately.

V = current pooling window; n = number of entries in that window. Pooling learns no weights or biases.

y_{\max}=\max VLaTeX: y_{\max}=\max V
y_{\mathrm{avg}}=\frac1n\sum_{v\in V}vLaTeX: y_{\mathrm{avg}}=\frac1n\sum_{v\in V}v

Yaad rakhne ka cue: Max mein biggest; average mein total ÷ window ka count.

Solution — step by step

Formulas and meaning

Each 2 × 2 window contains four numbers. Max pooling selects the largest of those four. Average pooling adds all four and divides by 4. Move the window by the given stride after producing one answer.

p_{\mathrm{max}}=\max(a_1,a_2,a_3,a_4)LaTeX: p_{\mathrm{max}}=\max(a_1,a_2,a_3,a_4)
p_{\mathrm{avg}}=\frac{a_1+a_2+a_3+a_4}{4}LaTeX: p_{\mathrm{avg}}=\frac{a_1+a_2+a_3+a_4}{4}
N_{\mathrm{pool}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{pool}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

Use the input size at the pooling layer. The output keeps the same channel count.

Shape: \left\lfloor\frac{4-2}{2}\right\rfloor+1LaTeX: \left\lfloor\frac{4-2}{2}\right\rfloor+1 = 2 on each axis; channels stay unchanged.

Pooling window (1,1)

Step 1: select the entries.

[1, 4, 0, 2]

Step 2: add all selected entries.

1 + 4 + 0 + 2 = 7

Step 3: divide by the 4 entries in this window.

7/4 = 1.75. Maximum = 4

Pooling window (1,2)

Step 1: select the entries.

[2, 3, 5, 1]

Step 2: add all selected entries.

2 + 3 + 5 + 1 = 11

Step 3: divide by the 4 entries in this window.

11/4 = 2.75. Maximum = 5

Pooling window (2,1)

Step 1: select the entries.

[6, 3, 1, 7]

Step 2: add all selected entries.

6 + 3 + 1 + 7 = 17

Step 3: divide by the 4 entries in this window.

17/4 = 4.25. Maximum = 7

Pooling window (2,2)

Step 1: select the entries.

[2, 0, 4, 8]

Step 2: add all selected entries.

2 + 0 + 4 + 8 = 14

Step 3: divide by the 4 entries in this window.

14/4 = 3.5. Maximum = 8

Average output: \begin{bmatrix}1.75 & 2.75\\4.25 & 3.5\end{bmatrix}LaTeX: \begin{bmatrix}1.75 & 2.75\\4.25 & 3.5\end{bmatrix}. Max output: \begin{bmatrix}4 & 5\\7 & 8\end{bmatrix}LaTeX: \begin{bmatrix}4 & 5\\7 & 8\end{bmatrix}. Pooling normally has zero trainable parameters and reduces spatial dimensions.

Drill 46 Mixed convolution pool dense and loss

Question

Question. A is shown below, kernel Matrix 2 shown below, stride 1, valid. Apply ReLU, then 2 × 2 max pool stride 1. Flatten rowwise, use weight vector [0.1,0.2,0.3,0.4], bias 0, sigmoid output, target 1. Find all stages and BCE.

Given matrices
Input A
1201
0132
2101
1023
Kernel K · Matrix 2
10
0−1

Stride 1; P = 0 (no padding).

Answer — step by step

Given

Input A and kernel Matrix 2 are printed with the question.

Conv: stride 1, valid padding; follow with ReLU.

Max pool: 2 × 2, stride 1; flatten row by row.

Dense weights = [0.1,0.2,0.3,0.4]; bias = 0; sigmoid output.

Binary target = 1.

To find

Find convolution, activated map, pooled map, flattened values, prediction and BCE.

Formula — which one and why

Convolution values. At each valid location, multiply corresponding window and kernel entries, add all products and add the filter bias once. Move by the stated stride and repeat.

X = input; K = kernel; S = stride; b = filter bias; u,v locate a kernel entry. Use the printed kernel without flipping under the stated deep-learning convention.

Y_{ij}=\sum_{u,v}X_{iS+u,jS+v}K_{uv}+bLaTeX: Y_{ij}=\sum_{u,v}X_{iS+u,jS+v}K_{uv}+b

Yaad rakhne ka cue: Matching cells multiply; sab products add; bias sirf ek baar.

Pooling. Max pooling selects the largest window entry. Average pooling adds its entries and divides by their count. Each channel is pooled separately.

V = current pooling window; n = number of entries in that window. Pooling learns no weights or biases.

y_{\max}=\max VLaTeX: y_{\max}=\max V
y_{\mathrm{avg}}=\frac1n\sum_{v\in V}vLaTeX: y_{\mathrm{avg}}=\frac1n\sum_{v\in V}v

Yaad rakhne ka cue: Max mein biggest; average mein total ÷ window ka count.

Weighted sum and activation. Each input is multiplied by its connection weight. Add these contributions, add the bias once, then apply the stated activation.

xᵢ = input i; wᵢ = its weight; b = bias; z = sum before activation; a = output after activation; f = the specified activation function.

z=\sum_iw_ix_i+bLaTeX: z=\sum_iw_ix_i+b
a=f(z)LaTeX: a=f(z)

Yaad rakhne ka cue: Input × weight, sabka sum, bias once, phir activation.

Sigmoid and its slope. Use sigmoid when the neuron is specified to squash its preactivation into (0,1). Once its output is known, its derivative needs only output × (1−output).

z = preactivation; p = sigmoid output; e is the exponential base, not a prediction error in this formula.

p=\frac{1}{1+e^{-z}}LaTeX: p=\frac{1}{1+e^{-z}}
\frac{dp}{dz}=p(1-p)LaTeX: \frac{dp}{dz}=p(1-p)

Yaad rakhne ka cue: Sigmoid output p; uska slope p × (1−p).

Binary cross-entropy. For a binary target, choose the probability of the correct outcome and take its negative natural log. Sigmoid plus BCE has combined logit derivative p−y.

y is 0 or 1; p is the predicted probability of class 1; z is the sigmoid logit. The p−y derivative already includes the sigmoid derivative.

L=-[y\ln p+(1-y)\ln(1-p)]LaTeX: L=-[y\ln p+(1-y)\ln(1-p)]
\frac{\partial L}{\partial z}=p-yLaTeX: \frac{\partial L}{\partial z}=p-y

Yaad rakhne ka cue: Binary target 1 ho toh −ln(p); target 0 ho toh −ln(1−p).

Solution — step by step

Formulas and the operation we are performing

A filter window produces one output value. Multiply entries at matching positions and add the products. In this deep-learning convention, keep the filter in its printed orientation.

Y_{ij}=\sum_{u=0}^{K_h-1}\sum_{v=0}^{K_w-1}X_{iS+u,jS+v}K_{uv}+bLaTeX: Y_{ij}=\sum_{u=0}^{K_h-1}\sum_{v=0}^{K_w-1}X_{iS+u,jS+v}K_{uv}+b

Here i and j identify an output position; u and v count positions inside its window. S is stride, and b is the filter bias, zero in this question. The equation uses a padded input if padding is present and zero-based indices; the working below labels output positions from 1 for readability.

N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

N is input width/height, P is padding per side, and K is filter width/height. Floor means take the integer below the result, then add one.

Formulas and meaning

Each 2 × 2 window contains four numbers. Max pooling selects the largest of those four. Average pooling adds all four and divides by 4. Move the window by the given stride after producing one answer.

p_{\mathrm{max}}=\max(a_1,a_2,a_3,a_4)LaTeX: p_{\mathrm{max}}=\max(a_1,a_2,a_3,a_4)
p_{\mathrm{avg}}=\frac{a_1+a_2+a_3+a_4}{4}LaTeX: p_{\mathrm{avg}}=\frac{a_1+a_2+a_3+a_4}{4}
N_{\mathrm{pool}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1LaTeX: N_{\mathrm{pool}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1

Use the input size at the pooling layer. The output keeps the same channel count.

Completing the network after pooling

Flatten row by row. Multiply the resulting entries by the dense weights and add bias. Apply sigmoid to produce p. For target 1, binary cross-entropy is minus ln(p).

z=\sum_iw_ix_i+b,\qquad p=\frac1{1+e^{-z}},\qquad L=-\ln pLaTeX: z=\sum_iw_ix_i+b,\qquad p=\frac1{1+e^{-z}},\qquad L=-\ln p

Follow each selected input window below. Matching row colors connect its input cells, kernel coefficients and products; the outlined output cell shows where their sum belongs.

Step 1 shape: \left\lfloor\frac{4+2 \times 0-2}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{4+2 \times 0-2}{1}\right\rfloor+1 = 3 along each axis. Use the deep-learning cross-correlation convention: do not flip the printed kernel.

Output position (1, 1)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (1, 1) into one output cell

Input rows 1–2, columns 1–2. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
10
0−1

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 11 + 0 = 1
Row 20 + (−1) = −1

1 + (−1) = 0

5. Place it in the output
0−1−2
−112
2−1−3

Output row 1, column 1.

Window row 1.

R_1=(1)\times(1)+(2)\times(0)=1LaTeX: R_1=(1)\times(1)+(2)\times(0)=1

Window row 2.

R_2=(0)\times(0)+(1)\times(-1)=-1LaTeX: R_2=(0)\times(0)+(1)\times(-1)=-1

Add the rows to obtain one output cell.

Y_{1,1}=(1)+(-1)=0LaTeX: Y_{1,1}=(1)+(-1)=0

Output position (1, 2)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (1, 2) into one output cell

Input rows 1–2, columns 2–3. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
20
0−3

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 12 + 0 = 2
Row 20 + (−3) = −3

2 + (−3) = −1

5. Place it in the output
0−1−2
−112
2−1−3

Output row 1, column 2.

Window row 1.

R_1=(2)\times(1)+(0)\times(0)=2LaTeX: R_1=(2)\times(1)+(0)\times(0)=2

Window row 2.

R_2=(1)\times(0)+(3)\times(-1)=-3LaTeX: R_2=(1)\times(0)+(3)\times(-1)=-3

Add the rows to obtain one output cell.

Y_{1,2}=(2)+(-3)=-1LaTeX: Y_{1,2}=(2)+(-3)=-1

Output position (1, 3)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (1, 3) into one output cell

Input rows 1–2, columns 3–4. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
00
0−2

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 10 + 0 = 0
Row 20 + (−2) = −2

0 + (−2) = −2

5. Place it in the output
0−1−2
−112
2−1−3

Output row 1, column 3.

Window row 1.

R_1=(0)\times(1)+(1)\times(0)=0LaTeX: R_1=(0)\times(1)+(1)\times(0)=0

Window row 2.

R_2=(3)\times(0)+(2)\times(-1)=-2LaTeX: R_2=(3)\times(0)+(2)\times(-1)=-2

Add the rows to obtain one output cell.

Y_{1,3}=(0)+(-2)=-2LaTeX: Y_{1,3}=(0)+(-2)=-2

Output position (2, 1)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (2, 1) into one output cell

Input rows 2–3, columns 1–2. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
00
0−1

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 10 + 0 = 0
Row 20 + (−1) = −1

0 + (−1) = −1

5. Place it in the output
0−1−2
−112
2−1−3

Output row 2, column 1.

Window row 1.

R_1=(0)\times(1)+(1)\times(0)=0LaTeX: R_1=(0)\times(1)+(1)\times(0)=0

Window row 2.

R_2=(2)\times(0)+(1)\times(-1)=-1LaTeX: R_2=(2)\times(0)+(1)\times(-1)=-1

Add the rows to obtain one output cell.

Y_{2,1}=(0)+(-1)=-1LaTeX: Y_{2,1}=(0)+(-1)=-1

Output position (2, 2)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (2, 2) into one output cell

Input rows 2–3, columns 2–3. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
10
00

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 11 + 0 = 1
Row 20 + 0 = 0

1 + 0 = 1

5. Place it in the output
0−1−2
−112
2−1−3

Output row 2, column 2.

Window row 1.

R_1=(1)\times(1)+(3)\times(0)=1LaTeX: R_1=(1)\times(1)+(3)\times(0)=1

Window row 2.

R_2=(1)\times(0)+(0)\times(-1)=0LaTeX: R_2=(1)\times(0)+(0)\times(-1)=0

Add the rows to obtain one output cell.

Y_{2,2}=(1)+(0)=1LaTeX: Y_{2,2}=(1)+(0)=1

Output position (2, 3)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (2, 3) into one output cell

Input rows 2–3, columns 3–4. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
30
0−1

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 13 + 0 = 3
Row 20 + (−1) = −1

3 + (−1) = 2

5. Place it in the output
0−1−2
−112
2−1−3

Output row 2, column 3.

Window row 1.

R_1=(3)\times(1)+(2)\times(0)=3LaTeX: R_1=(3)\times(1)+(2)\times(0)=3

Window row 2.

R_2=(0)\times(0)+(1)\times(-1)=-1LaTeX: R_2=(0)\times(0)+(1)\times(-1)=-1

Add the rows to obtain one output cell.

Y_{2,3}=(3)+(-1)=2LaTeX: Y_{2,3}=(3)+(-1)=2

Output position (3, 1)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (3, 1) into one output cell

Input rows 3–4, columns 1–2. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
20
00

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 12 + 0 = 2
Row 20 + 0 = 0

2 + 0 = 2

5. Place it in the output
0−1−2
−112
2−1−3

Output row 3, column 1.

Window row 1.

R_1=(2)\times(1)+(1)\times(0)=2LaTeX: R_1=(2)\times(1)+(1)\times(0)=2

Window row 2.

R_2=(1)\times(0)+(0)\times(-1)=0LaTeX: R_2=(1)\times(0)+(0)\times(-1)=0

Add the rows to obtain one output cell.

Y_{3,1}=(2)+(0)=2LaTeX: Y_{3,1}=(2)+(0)=2

Output position (3, 2)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (3, 2) into one output cell

Input rows 3–4, columns 2–3. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
10
0−2

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 11 + 0 = 1
Row 20 + (−2) = −2

1 + (−2) = −1

5. Place it in the output
0−1−2
−112
2−1−3

Output row 3, column 2.

Window row 1.

R_1=(1)\times(1)+(0)\times(0)=1LaTeX: R_1=(1)\times(1)+(0)\times(0)=1

Window row 2.

R_2=(0)\times(0)+(2)\times(-1)=-2LaTeX: R_2=(0)\times(0)+(2)\times(-1)=-2

Add the rows to obtain one output cell.

Y_{3,2}=(1)+(-2)=-1LaTeX: Y_{3,2}=(1)+(-2)=-1

Output position (3, 3)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (3, 3) into one output cell

Input rows 3–4, columns 3–4. The window moves 1 cell per output step.

1. Select the input window
1201
0132
2101
1023

Only the outlined cells belong to this output.

2. Match the kernel
10
0−1

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell
00
0−3

Each cell is input × matching kernel coefficient.

4. Add the products row by row
Row 10 + 0 = 0
Row 20 + (−3) = −3

0 + (−3) = −3

5. Place it in the output
0−1−2
−112
2−1−3

Output row 3, column 3.

Window row 1.

R_1=(0)\times(1)+(1)\times(0)=0LaTeX: R_1=(0)\times(1)+(1)\times(0)=0

Window row 2.

R_2=(2)\times(0)+(3)\times(-1)=-3LaTeX: R_2=(2)\times(0)+(3)\times(-1)=-3

Add the rows to obtain one output cell.

Y_{3,3}=(0)+(-3)=-3LaTeX: Y_{3,3}=(0)+(-3)=-3

Final output: \begin{bmatrix}0 & -1 & -2\\-1 & 1 & 2\\2 & -1 & -3\end{bmatrix}LaTeX: \begin{bmatrix}0 & -1 & -2\\-1 & 1 & 2\\2 & -1 & -3\end{bmatrix}. Each output came from its own complete window; stride changes the next window position, not the multiplication rule.

ReLU turns negative entries into 0: \begin{bmatrix}0 & 0 & 0\\0 & 1 & 2\\2 & 0 & 0\end{bmatrix}LaTeX: \begin{bmatrix}0 & 0 & 0\\0 & 1 & 2\\2 & 0 & 0\end{bmatrix}.

Shape: \left\lfloor\frac{3-2}{1}\right\rfloor+1LaTeX: \left\lfloor\frac{3-2}{1}\right\rfloor+1 = 2 on each axis; channels stay unchanged.

Pooling window (1,1)

Step 1: select the entries.

[0, 0, 0, 1]

Step 2: add all selected entries.

0 + 0 + 0 + 1 = 1

Step 3: divide by the 4 entries in this window.

1/4 = 0.25. Maximum = 1

Pooling window (1,2)

Step 1: select the entries.

[0, 0, 1, 2]

Step 2: add all selected entries.

0 + 0 + 1 + 2 = 3

Step 3: divide by the 4 entries in this window.

3/4 = 0.75. Maximum = 2

Pooling window (2,1)

Step 1: select the entries.

[0, 1, 2, 0]

Step 2: add all selected entries.

0 + 1 + 2 + 0 = 3

Step 3: divide by the 4 entries in this window.

3/4 = 0.75. Maximum = 2

Pooling window (2,2)

Step 1: select the entries.

[1, 2, 0, 0]

Step 2: add all selected entries.

1 + 2 + 0 + 0 = 3

Step 3: divide by the 4 entries in this window.

3/4 = 0.75. Maximum = 2

Average output: \begin{bmatrix}0.25 & 0.75\\0.75 & 0.75\end{bmatrix}LaTeX: \begin{bmatrix}0.25 & 0.75\\0.75 & 0.75\end{bmatrix}. Max output: \begin{bmatrix}1 & 2\\2 & 2\end{bmatrix}LaTeX: \begin{bmatrix}1 & 2\\2 & 2\end{bmatrix}. Pooling normally has zero trainable parameters and reduces spatial dimensions.

Use MAX output and flatten rowwise: [1, 2, 2, 2]. Dense z = 0.1 × 1 + 0.2 × 2 + 0.3 × 2 + 0.4 × 2 = 1.9. p = σ(1.9) = 0.869891526. BCE = −ln(p) = 0.139386758. Average pool values were computed for contrast; the question requires max pool.