# GRU state under the retain-old convention

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/formulas/45

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/45/

Some GRU questions use the opposite naming convention: z is the fraction of the OLD state retained. The formula below uses that convention. Do not combine its formula with an explanation that says z retains the new candidate.



$$
\widetilde h_t=\tanh(W_hx_t+U_h(r_t\odot h_{t-1})+b_h)
$$





$$
h_t=z_t\odot h_{t-1}+(1-z_t)\odot\widetilde h_t
$$



**Question. Use the retain-old GRU convention. Given hprevious = 0.4, reset gate r = 0.5, update gate z = 0.25, x = 1, W = 0.2, U = 0.5 and b = 0, calculate the candidate and the next hidden state.**

### Given

h\_old = 0.4; r = 0.5; z = 0.25; x = 1.

W = 0.2; U = 0.5; b = 0; z retains the old state.

### To find

Find the candidate and new hidden state.

### Formula — which one and why

**GRU with a retain-old update gate.** Form the candidate using reset memory; then z retains the old state and 1−z admits the candidate. Follow this question-specific convention.

r = reset gate; z = retain-old gate; h\_old = previous state; h̃ = candidate.



$$
\widetilde h=\tanh(Wx+U(rh_{\mathrm{old}})+b)
$$





$$
h_{\mathrm{new}}=zh_{\mathrm{old}}+(1-z)\widetilde h
$$



**Yaad rakhne ka cue:** Question mein z kisko multiply karta hai wahi meaning; yahan old ko retain.

### Solution — step by step

**Step 1: reset the old state for candidate formation.**



$$
r h_{\mathrm{previous}}=0.5(0.4)=0.2
$$



**Step 2: calculate the candidate's raw input.** Its new-input contribution is 0.2 × 1; its reset-memory contribution is 0.5 × 0.2.



$$
a=0.2(1)+0.5(0.2)+0=0.2+0.1=0.3
$$



**Step 3: apply tanh to this raw input.**



$$
\widetilde h=\tanh(0.3)\approx0.291312612
$$



**Step 4: calculate how much old state is retained.** In this question, z = 0.25 multiplies the original old state 0.4.



$$
z h_{\mathrm{previous}}=0.25(0.4)=0.1
$$



**Step 5: calculate how much candidate is admitted.** The other fraction is 1 − z = 0.75.



$$
1-z=1-0.25=0.75
$$





$$
 (1-z)\widetilde h=0.75(0.291312612)\approx0.218484459
$$



**Step 6: add the two final contributions.**



$$
h=0.1+0.218484459=0.318484459
$$



**Final answer: h ≈ 0.318484459.** The candidate was 0.291312612; the final hidden state is a mixture of that candidate and the original old state. Under the admit-new convention with the same numeric z, the result would instead be 0.372828153.

### Retain-old GRU with gate 0

**Question. Use h\_new = z h\_old+(1−z) candidate. Given h\_old = 0.4, z = 0, r = 0.5, x = 1, W = 0.2, U = 0.5 and b = 0, calculate the candidate and new state.**

### Given

h\_old = 0.4; z = 0; r = 0.5; x = 1.

W = 0.2; U = 0.5; b = 0; z is the retain-old gate.

### To find

Candidate and final hidden state.

### Formula — which one and why

The reset gate changes old-state input to the candidate. The update gate separately mixes the original old state and candidate.

The question explicitly makes z multiply the old state. r and z do different jobs.



$$
\widetilde h=\tanh(Wx+U(rh_{\mathrm{old}})+b)
$$





$$
h_{\mathrm{new}}=zh_{\mathrm{old}}+(1-z)\widetilde h
$$



**Yaad rakhne ka cue:** Reset sirf candidate mein; update mix mein original old state.

### Solution — step by step

**Step 1: reset the old state for candidate formation.**



$$
rh_{\mathrm{old}}=0.5(0.4)=0.2
$$



**Step 2: calculate the candidate raw sum.**



$$
a=0.2(1)+0.5(0.2)+0=0.3
$$



**Step 3: apply tanh.**



$$
\widetilde h=\tanh(0.3)\approx0.291312612
$$



**Step 4: retain the old-state fraction.**



$$
0(0.4)=0
$$



**Step 5: calculate the candidate fraction.**



$$
(1-0)(0.291312612)\approx0.291312612
$$



**Step 6: add the two fractions.**



$$
h_{\mathrm{new}}=0+(0.291312612)\approx0.291312612
$$



**Final answer:** Candidate ≈ 0.291312612; new state ≈ 0.291312612.

### Retain-old GRU with gate 0.8

**Question. Use h\_new = z h\_old+(1−z) candidate. Given h\_old = -0.2, z = 0.8, r = 0.5, x = 1, W = 0.2, U = 0.5 and b = 0, calculate the candidate and new state.**

### Given

h\_old = -0.2; z = 0.8; r = 0.5; x = 1.

W = 0.2; U = 0.5; b = 0; z is the retain-old gate.

### To find

Candidate and final hidden state.

### Formula — which one and why

The reset gate changes old-state input to the candidate. The update gate separately mixes the original old state and candidate.

The question explicitly makes z multiply the old state. r and z do different jobs.



$$
\widetilde h=\tanh(Wx+U(rh_{\mathrm{old}})+b)
$$





$$
h_{\mathrm{new}}=zh_{\mathrm{old}}+(1-z)\widetilde h
$$



**Yaad rakhne ka cue:** Reset sirf candidate mein; update mix mein original old state.

### Solution — step by step

**Step 1: reset the old state for candidate formation.**



$$
rh_{\mathrm{old}}=0.5(-0.2)=-0.1
$$



**Step 2: calculate the candidate raw sum.**



$$
a=0.2(1)+0.5(-0.1)+0=0.15
$$



**Step 3: apply tanh.**



$$
\widetilde h=\tanh(0.15)\approx0.148885034
$$



**Step 4: retain the old-state fraction.**



$$
0.8(-0.2)=-0.16
$$



**Step 5: calculate the candidate fraction.**



$$
(1-0.8)(0.148885034)\approx0.029777007
$$



**Step 6: add the two fractions.**



$$
h_{\mathrm{new}}=-0.16+(0.029777007)\approx-0.130222993
$$



**Final answer:** Candidate ≈ 0.148885034; new state ≈ -0.130222993.