Drill 44 GRU reset then mix

Answer — step by step

Given

Old hidden state = 0.4; reset r = 0.5; update z = 0.25.

x = 1; W = 0.2; U = 0.5; b = 0.

Use h = (1−z)h_old + z×candidate.

To find

Find reset contribution, candidate and new hidden state.

Formula — which one and why

GRU reset and state mixture. The reset gate controls old-state information entering the candidate. The stated update convention then mixes the original old state and the candidate.

r = reset gate; z = update gate; h_old = previous state; h_tilde = candidate. Here z weights the candidate; some questions instead define z to weight the old state.

LaTeX: \widetilde h=\tanh(Wx+U(r\odot h_{\mathrm{old}})+b)

LaTeX: h=(1-z)\odot h_{\mathrm{old}}+z\odot\widetilde h

Yaad rakhne ka cue: Reset candidate ko control; update gate old-new mixture ko control.

Solution — step by step

Meaning and given convention

The reset gate r scales the old state while making the candidate. The update gate z controls the amount of candidate used in the final mixture under this question's convention. Some books reverse the roles of z and 1 − z, so copy the supplied formula.

LaTeX: \widetilde h=\tanh\left(Wx+U(rh_{\mathrm{previous}})+b\right)

LaTeX: h=(1-z)h_{\mathrm{previous}}+z\widetilde h

Step 1: reset the old state for candidate construction

LaTeX: rh_{\mathrm{previous}}=0.5\times0.4=0.2

Step 2: form the candidate's raw score

LaTeX: Wx=0.2\times1=0.2

LaTeX: U(rh_{\mathrm{previous}})=0.5\times0.2=0.1

LaTeX: a=0.2+0.1+0=0.3

Step 3: apply tanh

LaTeX: \widetilde h=\tanh(0.3)=0.291312612

Step 4: mix the original old state with the candidate

LaTeX: 1-z=1-0.25=0.75

LaTeX: h=0.75(0.4)+0.25(0.291312612)

LaTeX: h=0.3+0.072828153=0.372828153

Final answer: h ≈ 0.372828153. The final old-state term uses 0.4, not the reset value 0.2.