# Drill 40 Mixed convolution pool dense and loss

Subject: Deep Learning

Exam: MT2

Reader: https://deep-learning-study-room.vercel.app/#mt2/read/17/mt2-c17-s40

Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c17-s40/

## Question

**Question.** A is shown below, kernel Matrix 2 shown below, stride 1, valid. Apply ReLU, then 2 × 2 max pool stride 1. Flatten rowwise, use weight vector [0.1,0.2,0.3,0.4], bias 0, sigmoid output, target 1. Find all stages and BCE.

Given matrices

Input A

Input A

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Kernel K · Matrix 2

Kernel K · Matrix 2

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Stride 1; P = 0 (no padding).

## Answer — step by step

### Given

Input A and kernel Matrix 2 are printed with the question.

Conv: stride 1, valid padding; follow with ReLU.

Max pool: 2 × 2, stride 1; flatten row by row.

Dense weights = [0.1,0.2,0.3,0.4]; bias = 0; sigmoid output.

Binary target = 1.

### To find

Find convolution, activated map, pooled map, flattened values, prediction and BCE.

### Formula — which one and why

**Convolution values.** At each valid location, multiply corresponding window and kernel entries, add all products and add the filter bias once. Move by the stated stride and repeat.

X = input; K = kernel; S = stride; b = filter bias; u,v locate a kernel entry. Use the printed kernel without flipping under the stated deep-learning convention.



$$
Y_{ij}=\sum_{u,v}X_{iS+u,jS+v}K_{uv}+b
$$



**Yaad rakhne ka cue:** Matching cells multiply; sab products add; bias sirf ek baar.

**Pooling.** Max pooling selects the largest window entry. Average pooling adds its entries and divides by their count. Each channel is pooled separately.

V = current pooling window; n = number of entries in that window. Pooling learns no weights or biases.



$$
y_{\max}=\max V
$$





$$
y_{\mathrm{avg}}=\frac1n\sum_{v\in V}v
$$



**Yaad rakhne ka cue:** Max mein biggest; average mein total ÷ window ka count.

**Weighted sum and activation.** Each input is multiplied by its connection weight. Add these contributions, add the bias once, then apply the stated activation.

xᵢ = input i; wᵢ = its weight; b = bias; z = sum before activation; a = output after activation; f = the specified activation function.



$$
z=\sum_iw_ix_i+b
$$





$$
a=f(z)
$$



**Yaad rakhne ka cue:** Input × weight, sabka sum, bias once, phir activation.

**Sigmoid and its slope.** Use sigmoid when the neuron is specified to squash its preactivation into (0,1). Once its output is known, its derivative needs only output × (1−output).

z = preactivation; p = sigmoid output; e is the exponential base, not a prediction error in this formula.



$$
p=\frac{1}{1+e^{-z}}
$$





$$
\frac{dp}{dz}=p(1-p)
$$



**Yaad rakhne ka cue:** Sigmoid output p; uska slope p × (1−p).

**Binary cross-entropy.** For a binary target, choose the probability of the correct outcome and take its negative natural log. Sigmoid plus BCE has combined logit derivative p−y.

y is 0 or 1; p is the predicted probability of class 1; z is the sigmoid logit. The p−y derivative already includes the sigmoid derivative.



$$
L=-[y\ln p+(1-y)\ln(1-p)]
$$





$$
\frac{\partial L}{\partial z}=p-y
$$



**Yaad rakhne ka cue:** Binary target 1 ho toh −ln(p); target 0 ho toh −ln(1−p).

### Solution — step by step

### Formulas and the operation we are performing

A filter window produces one output value. Multiply entries at matching positions and add the products. In this deep-learning convention, keep the filter in its printed orientation.



$$
Y_{ij}=\sum_{u=0}^{K_h-1}\sum_{v=0}^{K_w-1}X_{iS+u,jS+v}K_{uv}+b
$$



Here i and j identify an output position; u and v count positions inside its window. S is stride, and b is the filter bias, zero in this question. The equation uses a padded input if padding is present and zero-based indices; the working below labels output positions from 1 for readability.



$$
N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1
$$



N is input width/height, P is padding per side, and K is filter width/height. Floor means take the integer below the result, then add one.

### Formulas and meaning

Each 2 × 2 window contains four numbers. Max pooling selects the largest of those four. Average pooling adds all four and divides by 4. Move the window by the given stride after producing one answer.



$$
p_{\mathrm{max}}=\max(a_1,a_2,a_3,a_4)
$$





$$
p_{\mathrm{avg}}=\frac{a_1+a_2+a_3+a_4}{4}
$$





$$
N_{\mathrm{pool}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1
$$



Use the input size at the pooling layer. The output keeps the same channel count.

### Completing the network after pooling

Flatten row by row. Multiply the resulting entries by the dense weights and add bias. Apply sigmoid to produce p. For target 1, binary cross-entropy is minus ln(p).



$$
z=\sum_iw_ix_i+b,\qquad p=\frac1{1+e^{-z}},\qquad L=-\ln p
$$



Follow each selected input window below. Matching row colors connect its input cells, kernel coefficients and products; the outlined output cell shows where their sum belongs.

**Step 1 shape:**   $\left\lfloor\frac{4+2 \times  0-2}{1}\right\rfloor+1$  = 3 along each axis. Use the deep-learning cross-correlation convention: do not flip the printed kernel.

### Output position (1, 1)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (1, 1) into one output cell

Input rows **1–2**, columns **1–2**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**1 + 0 = 1**

Row 2**0 + (−1) = −1**

**1 + (−1) = 0**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 1, column 1.

**Window row 1.**



$$
R_1=(1)\times(1)+(2)\times(0)=1
$$



**Window row 2.**



$$
R_2=(0)\times(0)+(1)\times(-1)=-1
$$



**Add the rows to obtain one output cell.**



$$
Y_{1,1}=(1)+(-1)=0
$$



### Output position (1, 2)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (1, 2) into one output cell

Input rows **1–2**, columns **2–3**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 2 | 0 |
| 0 | −3 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**2 + 0 = 2**

Row 2**0 + (−3) = −3**

**2 + (−3) = −1**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 1, column 2.

**Window row 1.**



$$
R_1=(2)\times(1)+(0)\times(0)=2
$$



**Window row 2.**



$$
R_2=(1)\times(0)+(3)\times(-1)=-3
$$



**Add the rows to obtain one output cell.**



$$
Y_{1,2}=(2)+(-3)=-1
$$



### Output position (1, 3)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (1, 3) into one output cell

Input rows **1–2**, columns **3–4**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 0 | 0 |
| 0 | −2 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**0 + 0 = 0**

Row 2**0 + (−2) = −2**

**0 + (−2) = −2**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 1, column 3.

**Window row 1.**



$$
R_1=(0)\times(1)+(1)\times(0)=0
$$



**Window row 2.**



$$
R_2=(3)\times(0)+(2)\times(-1)=-2
$$



**Add the rows to obtain one output cell.**



$$
Y_{1,3}=(0)+(-2)=-2
$$



### Output position (2, 1)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (2, 1) into one output cell

Input rows **2–3**, columns **1–2**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 0 | 0 |
| 0 | −1 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**0 + 0 = 0**

Row 2**0 + (−1) = −1**

**0 + (−1) = −1**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 2, column 1.

**Window row 1.**



$$
R_1=(0)\times(1)+(1)\times(0)=0
$$



**Window row 2.**



$$
R_2=(2)\times(0)+(1)\times(-1)=-1
$$



**Add the rows to obtain one output cell.**



$$
Y_{2,1}=(0)+(-1)=-1
$$



### Output position (2, 2)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (2, 2) into one output cell

Input rows **2–3**, columns **2–3**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | 0 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**1 + 0 = 1**

Row 2**0 + 0 = 0**

**1 + 0 = 1**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 2, column 2.

**Window row 1.**



$$
R_1=(1)\times(1)+(3)\times(0)=1
$$



**Window row 2.**



$$
R_2=(1)\times(0)+(0)\times(-1)=0
$$



**Add the rows to obtain one output cell.**



$$
Y_{2,2}=(1)+(0)=1
$$



### Output position (2, 3)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (2, 3) into one output cell

Input rows **2–3**, columns **3–4**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 3 | 0 |
| 0 | −1 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**3 + 0 = 3**

Row 2**0 + (−1) = −1**

**3 + (−1) = 2**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 2, column 3.

**Window row 1.**



$$
R_1=(3)\times(1)+(2)\times(0)=3
$$



**Window row 2.**



$$
R_2=(0)\times(0)+(1)\times(-1)=-1
$$



**Add the rows to obtain one output cell.**



$$
Y_{2,3}=(3)+(-1)=2
$$



### Output position (3, 1)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (3, 1) into one output cell

Input rows **3–4**, columns **1–2**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 2 | 0 |
| 0 | 0 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**2 + 0 = 2**

Row 2**0 + 0 = 0**

**2 + 0 = 2**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 3, column 1.

**Window row 1.**



$$
R_1=(2)\times(1)+(1)\times(0)=2
$$



**Window row 2.**



$$
R_2=(1)\times(0)+(0)\times(-1)=0
$$



**Add the rows to obtain one output cell.**



$$
Y_{3,1}=(2)+(0)=2
$$



### Output position (3, 2)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (3, 2) into one output cell

Input rows **3–4**, columns **2–3**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −2 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**1 + 0 = 1**

Row 2**0 + (−2) = −2**

**1 + (−2) = −1**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 3, column 2.

**Window row 1.**



$$
R_1=(1)\times(1)+(0)\times(0)=1
$$



**Window row 2.**



$$
R_2=(0)\times(0)+(2)\times(-1)=-2
$$



**Add the rows to obtain one output cell.**



$$
Y_{3,2}=(1)+(-2)=-1
$$



### Output position (3, 3)

Use this position's input window. Multiply corresponding entries row by row; then add all row totals.

Follow window (3, 3) into one output cell

Input rows **3–4**, columns **3–4**. The window moves **1** cell per output step.

1. Select the input window

1. Select the input window

|  |  |  |  |
| --- | --- | --- | --- |
| 1 | 2 | 0 | 1 |
| 0 | 1 | 3 | 2 |
| 2 | 1 | 0 | 1 |
| 1 | 0 | 2 | 3 |

Only the outlined cells belong to this output.

2. Match the kernel

2. Match the kernel

|  |  |
| --- | --- |
| 1 | 0 |
| 0 | −1 |

Same row and column positions; no matrix multiplication.

3. Multiply cell by cell

3. Multiply cell by cell

|  |  |
| --- | --- |
| 0 | 0 |
| 0 | −3 |

Each cell is input × matching kernel coefficient.

4. Add the products row by row

Row 1**0 + 0 = 0**

Row 2**0 + (−3) = −3**

**0 + (−3) = −3**

5. Place it in the output

5. Place it in the output

|  |  |  |
| --- | --- | --- |
| 0 | −1 | −2 |
| −1 | 1 | 2 |
| 2 | −1 | −3 |

Output row 3, column 3.

**Window row 1.**



$$
R_1=(0)\times(1)+(1)\times(0)=0
$$



**Window row 2.**



$$
R_2=(2)\times(0)+(3)\times(-1)=-3
$$



**Add the rows to obtain one output cell.**



$$
Y_{3,3}=(0)+(-3)=-3
$$



**Final output:**   $\begin{bmatrix}0 & -1 & -2\\-1 & 1 & 2\\2 & -1 & -3\end{bmatrix}$ . Each output came from its own complete window; stride changes the next window position, not the multiplication rule.

ReLU turns negative entries into 0:  $\begin{bmatrix}0 & 0 & 0\\0 & 1 & 2\\2 & 0 & 0\end{bmatrix}$ .

**Shape:**   $\left\lfloor\frac{3-2}{1}\right\rfloor+1$  = 2 on each axis; channels stay unchanged.

### Pooling window (1,1)

**Step 1: select the entries.**

[0, 0, 0, 1]

**Step 2: add all selected entries.**

0 + 0 + 0 + 1 = 1

**Step 3: divide by the 4 entries in this window.**

1/4 = **0.25**. Maximum = **1**

### Pooling window (1,2)

**Step 1: select the entries.**

[0, 0, 1, 2]

**Step 2: add all selected entries.**

0 + 0 + 1 + 2 = 3

**Step 3: divide by the 4 entries in this window.**

3/4 = **0.75**. Maximum = **2**

### Pooling window (2,1)

**Step 1: select the entries.**

[0, 1, 2, 0]

**Step 2: add all selected entries.**

0 + 1 + 2 + 0 = 3

**Step 3: divide by the 4 entries in this window.**

3/4 = **0.75**. Maximum = **2**

### Pooling window (2,2)

**Step 1: select the entries.**

[1, 2, 0, 0]

**Step 2: add all selected entries.**

1 + 2 + 0 + 0 = 3

**Step 3: divide by the 4 entries in this window.**

3/4 = **0.75**. Maximum = **2**

**Average output:**   $\begin{bmatrix}0.25 & 0.75\\0.75 & 0.75\end{bmatrix}$ . **Max output:**   $\begin{bmatrix}1 & 2\\2 & 2\end{bmatrix}$ . Pooling normally has zero trainable parameters and reduces spatial dimensions.

Use MAX output and flatten rowwise: [1, 2, 2, 2]. Dense z = 0.1 × 1 + 0.2 × 2 + 0.3 × 2 + 0.4 × 2 = 1.9. p = σ(1.9) = 0.869891526. BCE = −ln(p) = **0.139386758**. Average pool values were computed for contrast; the question requires max pool.