# Drill 42 LSTM with a negative candidate

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s42

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s42/

## Question

**Question.** f = 0.8, i = 0.2, o = 0.6, candidate = −0.5, Cprevious = 0.3. Find C, h.

## Answer — step by step

### Given

Activated gates: f = 0.8, i = 0.2, o = 0.6.

Candidate = −0.5; old cell C = 0.3.

### To find

Find the new cell and hidden state.

### Formula — which one and why

**LSTM cell and hidden state.** Multiply old memory by the forget gate; multiply candidate content by the input gate; add those two contributions. Apply tanh to the new cell and multiply by the output gate.

f,i,o = activated forget/input/output gates; C\_old = previous cell; C\_tilde = candidate content; C = new cell; h = visible hidden state. ⊙ means elementwise multiplication.



$$
C=f\odot C_{\mathrm{old}}+i\odot\widetilde C
$$





$$
h=o\odot\tanh C
$$



**Yaad rakhne ka cue:** Purana rakho + naya lo; phir tanh aur output gate.

### Solution — step by step

### Meaning and formulas

An LSTM keeps a cell memory C and a visible hidden state h. The forget gate f controls old memory, the input gate i controls new candidate content, and the output gate o controls the visible state. These gate values are already supplied.



$$
C=fC_{\mathrm{previous}}+i\widetilde C,\qquad h=o\tanh(C)
$$



The candidate, written C with a tilde, may be negative. A negative candidate subtracts content when admitted.

### Step 1: retained old memory



$$
fC_{\mathrm{previous}}=0.8\times0.3=0.24
$$



### Step 2: admitted new content



$$
i\widetilde C=0.2\times(-0.5)=-0.1
$$



### Step 3: add them to obtain the new cell memory



$$
C=0.24+(-0.1)=0.14
$$



### Step 4: apply tanh to the cell memory



$$
\tanh(0.14)=\frac{e^{0.28}-1}{e^{0.28}+1}=0.139092448
$$



### Step 5: apply the output gate



$$
h=0.6\times0.139092448=0.083455469
$$



**Final answer:** cell memory 0.14; hidden state approximately 0.083455469. C and h are different quantities.