# A complete vector RNN calculation

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/9/tee-c9-s5

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c9-s5/

**Question from R26 Q5a.** xₜ = [0,1], hₜ₋₁ = [0,1], no bias is supplied, Wxh is shown below, Whh is shown below. Use tanh and rows as receiving neurons.

### Given

Input x = [0,1]ᵀ; previous h = [0,1]ᵀ.

Wxh =  $\begin{bmatrix}0.6 & 0.4\\0.3 & 0.7\end{bmatrix}$ .

Whh =  $\begin{bmatrix}0.5 & 0.3\\0.2 & 0.4\end{bmatrix}$ ; bias = 0.

Apply tanh separately to the two sums.

### To find

Find the new two-component hidden state; output-layer parameters are not supplied.

### Formula — which one and why

**Recurrent forward pass.** At each time step, combine the current input contribution with the previous activated hidden state, add the bias and apply tanh. The same weights are reused at every time step.

x\_t = current input; h\_(t−1) = previous hidden state; W\_xh = input weights; W\_hh = recurrent weights; b = bias.



$$
a_t=W_{xh}x_t+W_{hh}h_{t-1}+b
$$





$$
h_t=\tanh(a_t)
$$



**Yaad rakhne ka cue:** Naya input + pichhla activated state; phir activation.

### Solution — step by step

![Given matrices for this question](https://deep-learning-study-room.vercel.app/assets/figures/given_deac39f14c.svg)

Diagram: Given matrices for this question. Labels: GIVEN VALUES — read each matrix row from left to right | Wxh | 0.6 | 0.4 | 0.3 | 0.7 | Whh | 0.5 | 0.3 | 0.2 | 0.4

Given matrices for this question Tap the diagram to enlarge.

**Step 1 new input:**  Wxh x = [0.6 × 0 + 0.4 × 1, 0.3 × 0 + 0.7 × 1] = [0.4,0.7].

**Step 2 old memory:**  Whh hprevious = [0.5 × 0 + 0.3 × 1, 0.2 × 0 + 0.4 × 1] = [0.3,0.4].

**Step 3 add:**  a = [0.4+0.3, 0.7+0.4] = [0.7,1.1].

**Step 4 activate each element:**  h = [tanh(0.7), tanh(1.1)] = **[0.604368,0.800499]**. An output-layer prediction cannot be numerically calculated without its weights and bias.