# R25 Q7a · Long-text sentiment architecture

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/21/tee-c21-s14

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c21-s14/

## Question

**Question. Choose a deep architecture for long social-media/customer-review sequences classified positive,negative or neutral. Explain its operation with equations.** (10 marks)

## Answer — step by step

An LSTM is a suitable syllabus-aligned choice for long dependencies. Tokenize and embed the text, process the sequence through LSTM states, form a sequence representation and use a three-class softmax. Padding/masking must prevent padded tokens from being treated as content.

Text tokens → Embedding sequence → LSTM hidden/cell recurrence → Final or pooled representation → Dense3 +softmax → Positive / negative / neutral

**Definition.** Long short-term memory (LSTM) is a gated recurrent architecture with a cell state and hidden state. Forget, input and output gates regulate state retention, incorporation of candidate information and exposure of the updated cell.

### Explanation and working

Long short-term memory, or LSTM, keeps a cell state Cₜ and a hidden state hₜ. A sigmoid gate produces values from 0 to 1: approximately 0 blocks a component, approximately 1 passes it. Intermediate values pass a fraction. Gates are learned from the previous hidden state and current input, not manually assigned by the user during normal inference.

![Follow the cell state from left to right then identify forget input and output gates](https://deep-learning-study-room.vercel.app/assets/figures/lstm.svg)

Diagram: Follow the cell state from left to right then identify forget input and output gates. Labels: C previous | × | + | C new | tanh | × | h | new | σ: f | σ: i | tanh: C~ | σ: o | × | [h previous, x current] | Each gate has its own learned weights and bias.

Follow the cell state from left to right then identify forget input and output gates Tap the diagram to enlarge.



$$
f_t=\sigma(W_f[h_{t-1},x_t]+b_f),\quad i_t=\sigma(W_i[h_{t-1},x_t]+b_i)
$$





$$
\widetilde C_t=\tanh(W_c[h_{t-1},x_t]+b_c)
$$





$$
C_t=f_t\odot C_{t-1}+i_t\odot\widetilde C_t
$$





$$
o_t=\sigma(W_o[h_{t-1},x_t]+b_o),\qquad h_t=o_t\odot\tanh(C_t)
$$



The brackets [h, x] concatenate two vectors: [a, b] joined with [c, d] gives [a, b, c, d]. Concatenation is not addition. f is the forget gate: how much old cell content remains. i is the input gate: how much new candidate enters. The candidate is proposed content. o is the output gate: how much of the activated cell becomes visible as the hidden state.



$$
p=\operatorname{softmax}(Wh+b),\qquad L=-\sum_{k=1}^3y_k\log p_k
$$



Train on labelled examples with cross-entropy and BPTT; evaluate on held-out reviews with class-wise precision/recall/F1. LSTM gates reduce long-path gradient decay but do not guarantee perfect memory. The three outputs are mutually exclusive classes for this question.