# Drill 5 Matrix rows receive inputs

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s5

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s5/

## Question

**Question.** For the weight matrix W shown below, x = [2,1], b = [0.5,−0.5], find Wx+b and ReLU.

![Given matrices for this question](https://deep-learning-study-room.vercel.app/assets/figures/given_ed1fcbfdf5.svg)

Diagram: Given matrices for this question. Labels: GIVEN VALUES — read each matrix row from left to right | W | 1 | 2 | −1 | 3

Given matrices for this question Tap the diagram to enlarge.

## Answer — step by step

### Given

W =  $\begin{bmatrix}1 & 2\\-1 & 3\end{bmatrix}$ .

Input column x = [2,1]ᵀ; bias column b = [0.5,−0.5]ᵀ.

Activation: ReLU on each result.

### To find

Find Wx+b and the activated output.

### Formula — which one and why

**Row–column multiplication.** Each receiving neuron uses one weight row. Multiply that row by the input column, add the products, then add that neuron’s bias.

Wⱼᵢ = weight from input i to receiver j; xᵢ = input i; bⱼ = bias of receiver j; zⱼ = its preactivation. A superscript T means transpose: exchange rows and columns.



$$
z_j=\sum_i W_{ji}x_i+b_j
$$



**Yaad rakhne ka cue:** Ek row × poora input column = ek receiving neuron ka answer.

**ReLU and Leaky ReLU.** ReLU returns zero for negative input and retains positive input. Leaky ReLU instead multiplies a negative input by its supplied slope.

z = preactivation; α (alpha) = supplied negative-side slope for Leaky ReLU. The derivative at zero needs an explicit convention.



$$
\operatorname{ReLU}(z)=\max(0,z)
$$





$$
\operatorname{LeakyReLU}(z)=\begin{cases}z&z\ge0\\\alpha z&z<0\end{cases}
$$



**Yaad rakhne ka cue:** Positive ko rehne do; negative ko zero. Leaky mein chhota negative rehne do.

### Solution — step by step

### Meaning and formula

Each row supplies the weights of one receiving neuron. Multiply that row by the input column, then add that neuron's own bias. The two biases are separate; do not add both to every row.



$$
z=Wx+b,\qquad a_j=\max(0,z_j)
$$





$$
W=\begin{bmatrix}1&2\\-1&3\end{bmatrix},\quad x=\begin{bmatrix}2\\1\end{bmatrix},\quad b=\begin{bmatrix}0.5\\-0.5\end{bmatrix}
$$



### Step 1: calculate receiving neuron 1 from row 1



$$
z_1=(1\times2)+(2\times1)+0.5
$$





$$
z_1=2+2+0.5=4.5
$$



### Step 2: calculate receiving neuron 2 from row 2



$$
z_2=((-1)\times2)+(3\times1)+(-0.5)
$$





$$
z_2=-2+3-0.5=0.5
$$



### Step 3: apply ReLU to each entry



$$
a_1=\max(0,4.5)=4.5,\qquad a_2=\max(0,0.5)=0.5
$$





$$
z=\begin{bmatrix}4.5\\0.5\end{bmatrix},\qquad a=\begin{bmatrix}4.5\\0.5\end{bmatrix}
$$



The shape is 2 rows by 1 column: one output for each receiving row.