Drill 39 Two RNN steps

Answer — step by step

Given

Initial h₀ = 0; input weight Wxh = 0.5; recurrent weight Whh = 0.2; b = 0.

Inputs: x₁ = 1, x₂ = 2; activation: tanh.

To find

Calculate h₁, then use it to calculate h₂.

Formula — which one and why

Recurrent forward pass. At each time step, combine the current input contribution with the previous activated hidden state, add the bias and apply tanh. The same weights are reused at every time step.

x_t = current input; h_(t−1) = previous hidden state; W_xh = input weights; W_hh = recurrent weights; b = bias.

LaTeX: a_t=W_{xh}x_t+W_{hh}h_{t-1}+b

LaTeX: h_t=\tanh(a_t)

Yaad rakhne ka cue: Naya input + pichhla activated state; phir activation.

Solution — step by step

Meaning and formulas

At time t, the RNN combines the current input xₜ and the previous hidden state hₜ₋₁. The same two weights are reused at both times.

LaTeX: a_t=W_{xh}x_t+W_{hh}h_{t-1}+b,\qquad h_t=\tanh(a_t)

The raw sum a is not the hidden state. Tanh converts that sum into the state that is carried forward.

LaTeX: \tanh(a)=\frac{e^{2a}-1}{e^{2a}+1}

Step 1: calculate the first raw sum

LaTeX: a_1=0.5(1)+0.2(0)+0=0.5

Step 2: activate it to obtain the first state

LaTeX: h_1=\tanh(0.5)=\frac{e^1-1}{e^1+1}

LaTeX: h_1=\frac{2.718281828-1}{2.718281828+1}=0.462117157

Step 3: use that state in the second raw sum

LaTeX: a_2=0.5(2)+0.2(0.462117157)+0

LaTeX: a_2=1+0.092423431=1.092423431

Step 4: activate the second sum

LaTeX: h_2=\tanh(1.092423431)=0.797760954

Final answer: h₁ ≈ 0.462117157; h₂ ≈ 0.797760954. Pass h₁ to the next step, not the unactivated value 0.5.