Deep Learning · TEE · section

The convolution calculation

Convolutional neural networks

Definition. A convolutional layer calculates local weighted sums with a spatially shared kernel and optional bias. Standard deep-learning implementations ordinarily use cross-correlation, without flipping the kernel; examination calculations must follow the stated convention.

Explanation and working

In deep-learning software, the operation normally called convolution is cross-correlation: use the kernel as written, without flipping it. This is the convention in this course’s CNN numericals and the supplied latest answer key. Mathematical convolution would rotate the kernel; state that convention if a question explicitly requests it.

Question. Apply a 2 × 2 kernel Matrix 1 shown below to the 3 × 3 image Matrix 2 shown below, with stride 1, no padding and zero bias.

Given

Input X = LaTeX: \begin{bmatrix}1 & 2 & 3\\4 & 5 & 6\\7 & 8 & 9\end{bmatrix}.

Kernel K = LaTeX: \begin{bmatrix}1 & 0\\0 & -1\end{bmatrix}.

Stride = 1; padding = 0; bias = 0; no kernel flip.

To find

Find the 2 × 2 output map.

Formula — which one and why

Convolution values. At each valid location, multiply corresponding window and kernel entries, add all products and add the filter bias once. Move by the stated stride and repeat.

X = input; K = kernel; S = stride; b = filter bias; u,v locate a kernel entry. Use the printed kernel without flipping under the stated deep-learning convention.

LaTeX: Y_{ij}=\sum_{u,v}X_{iS+u,jS+v}K_{uv}+b

Yaad rakhne ka cue: Matching cells multiply; sab products add; bias sirf ek baar.

Solution — step by step

Diagram: Given matrices for this question. Labels: GIVEN VALUES — read each matrix row from left to right | Matrix 1 | 1 | 0 | 0 | −1 | Matrix 2 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9

LaTeX: X=\begin{bmatrix}1&2&3\\4&5&6\\7&8&9\end{bmatrix},\quad K=\begin{bmatrix}1&0\\0&-1\end{bmatrix}

Step 1. Top-left window. 1(1) + 2(0) + 4(0) + 5(−1) = 1 − 5 = −4.

Step 2. Move right by one. 2(1) + 3(0) + 5(0) + 6(−1) = 2 − 6 = −4.

Step 3. Move down and return left. 4(1) + 5(0) + 7(0) + 8(−1) = 4 − 8 = −4.

Step 4. Bottom-right window. 5(1) + 6(0) + 8(0) + 9(−1) = 5 − 9 = −4. Output is LaTeX: \begin{bmatrix}-4 & -4\\-4 & -4\end{bmatrix}. If ReLU follows, that next layer changes all four entries to zero; do not silently apply it when it was not requested.

Diagram: Four windows produce four output cells. Labels: Step 1: start (1,1) | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | K = [1, 0] | [0, −1] | 1×1 + 2×0 + 4×0 | + 5×(−1) = −4 | Step 2: start (1,2) | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | K = [1, 0] | [0, −1] | 2×1 + 3×0 + 5×0 | + 6×(−1) = −4 | Step 3: start (2,1) | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | K = [1, 0] | [0, −1] | 4×1 + 5×0 + 7×0 | + 8×(−1) = −4 | Step 4: start (2,2) | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | K = [1, 0] | [0, −1] | 5×1 + 6×0 + 8×0 | + 9×(−1) = −4 | Output: [−4, −4; −4, −4]. Kernel is not flipped.