Drill 5 Matrix rows receive inputs

Answer — step by step

Given

W = LaTeX: \begin{bmatrix}1 & 2\\-1 & 3\end{bmatrix}.

Input column x = [2,1]ᵀ; bias column b = [0.5,−0.5]ᵀ.

Activation: ReLU on each result.

To find

Find Wx+b and the activated output.

Formula — which one and why

Row–column multiplication. Each receiving neuron uses one weight row. Multiply that row by the input column, add the products, then add that neuron’s bias.

Wⱼᵢ = weight from input i to receiver j; xᵢ = input i; bⱼ = bias of receiver j; zⱼ = its preactivation. A superscript T means transpose: exchange rows and columns.

LaTeX: z_j=\sum_i W_{ji}x_i+b_j

Yaad rakhne ka cue: Ek row × poora input column = ek receiving neuron ka answer.

ReLU and Leaky ReLU. ReLU returns zero for negative input and retains positive input. Leaky ReLU instead multiplies a negative input by its supplied slope.

z = preactivation; α (alpha) = supplied negative-side slope for Leaky ReLU. The derivative at zero needs an explicit convention.

LaTeX: \operatorname{ReLU}(z)=\max(0,z)

LaTeX: \operatorname{LeakyReLU}(z)=\begin{cases}z&z\ge0\\\alpha z&z<0\end{cases}

Yaad rakhne ka cue: Positive ko rehne do; negative ko zero. Leaky mein chhota negative rehne do.

Solution — step by step

Meaning and formula

Each row supplies the weights of one receiving neuron. Multiply that row by the input column, then add that neuron's own bias. The two biases are separate; do not add both to every row.

LaTeX: z=Wx+b,\qquad a_j=\max(0,z_j)

LaTeX: W=\begin{bmatrix}1&2\\-1&3\end{bmatrix},\quad x=\begin{bmatrix}2\\1\end{bmatrix},\quad b=\begin{bmatrix}0.5\\-0.5\end{bmatrix}

Step 1: calculate receiving neuron 1 from row 1

LaTeX: z_1=(1\times2)+(2\times1)+0.5

LaTeX: z_1=2+2+0.5=4.5

Step 2: calculate receiving neuron 2 from row 2

LaTeX: z_2=((-1)\times2)+(3\times1)+(-0.5)

LaTeX: z_2=-2+3-0.5=0.5

Step 3: apply ReLU to each entry

LaTeX: a_1=\max(0,4.5)=4.5,\qquad a_2=\max(0,0.5)=0.5

LaTeX: z=\begin{bmatrix}4.5\\0.5\end{bmatrix},\qquad a=\begin{bmatrix}4.5\\0.5\end{bmatrix}

The shape is 2 rows by 1 column: one output for each receiving row.