# Drill 41 Two RNN steps

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s41

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s41/

## Question

**Question.** h₀ = 0, Wxh = 0.5, Whh = 0.2, b = 0, x₁ = 1, x₂ = 2, tanh activation. Find h₁, h₂.

## Answer — step by step

### Given

Initial h₀ = 0; input weight Wxh = 0.5; recurrent weight Whh = 0.2; b = 0.

Inputs: x₁ = 1, x₂ = 2; activation: tanh.

### To find

Calculate h₁, then use it to calculate h₂.

### Formula — which one and why

**Recurrent forward pass.** At each time step, combine the current input contribution with the previous activated hidden state, add the bias and apply tanh. The same weights are reused at every time step.

x\_t = current input; h\_(t−1) = previous hidden state; W\_xh = input weights; W\_hh = recurrent weights; b = bias.



$$
a_t=W_{xh}x_t+W_{hh}h_{t-1}+b
$$





$$
h_t=\tanh(a_t)
$$



**Yaad rakhne ka cue:** Naya input + pichhla activated state; phir activation.

### Solution — step by step

### Meaning and formulas

At time t, the RNN combines the current input xₜ and the previous hidden state hₜ₋₁. The same two weights are reused at both times.



$$
a_t=W_{xh}x_t+W_{hh}h_{t-1}+b,\qquad h_t=\tanh(a_t)
$$



The raw sum a is not the hidden state. Tanh converts that sum into the state that is carried forward.



$$
\tanh(a)=\frac{e^{2a}-1}{e^{2a}+1}
$$



### Step 1: calculate the first raw sum



$$
a_1=0.5(1)+0.2(0)+0=0.5
$$



### Step 2: activate it to obtain the first state



$$
h_1=\tanh(0.5)=\frac{e^1-1}{e^1+1}
$$





$$
h_1=\frac{2.718281828-1}{2.718281828+1}=0.462117157
$$



### Step 3: use that state in the second raw sum



$$
a_2=0.5(2)+0.2(0.462117157)+0
$$





$$
a_2=1+0.092423431=1.092423431
$$



### Step 4: activate the second sum



$$
h_2=\tanh(1.092423431)=0.797760954
$$



**Final answer:** h₁ ≈ 0.462117157; h₂ ≈ 0.797760954. Pass h₁ to the next step, not the unactivated value 0.5.