Deep Learning · TEE · section

Gate logic without memorizing a story

Recurrent neural networks

In an LSTM, let the previous cell be 2, forget gate 0.5, input gate 0.25, candidate −0.4 and output gate 1. Calculate the retained part, admitted part, new cell and hidden output.

Diagram: The input gate multiplies candidate content. The output gate is applied after updating and activating the cell.. Labels: Keep the order of gate calculations straight | Keep the order of gate calculations straight | The cell equation specifies what each gate multiplies. | Forget path | Old cell × | forget gate | Input path | Candidate × | input gate | Cell update | Add those two | parts | Hidden output | tanh(new cell) | × output gate | The input gate multiplies the candidate, not simply the raw input x.

Step 1 retained content: 0.5 × 2 = 1.

Step 2 admitted content: 0.25 × (−0.4) = −0.1. A candidate can be negative even though the gate itself is between zero and one.

Step 3 cell: 1 + (−0.1) = 0.9.

Step 4 hidden: output gate × tanh(cell) = 1 × tanh(0.9) ≈ 0.716298. The hidden output is not simply 0.9. Explain this distinction once before attempting another LSTM numerical.