# Q7b · Naive Bayes sentiment

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r24/q7b

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r24--q7b/

Previous examination · complete worked answers

**Question. Train Naive Bayes on: positive “amaze wonderful great”; “happy happy great”; “amaze great happy disgust”; “great happy wonderful”. Negative “disgust sad terrible great”; “sad terrible sad happy”. Predict sentence 7: “great happy disgust great”.** [10 marks]

### Answer

### Given

| Training text | Class |
| --- | --- |
| amaze wonderful great | positive |
| happy happy great | positive |
| amaze great happy disgust | positive |
| great happy wonderful | positive |
| disgust sad terrible great | negative |
| sad terrible sad happy | negative |

Test: great happy disgust great

### To find

Classify the test using multinomial Naive Bayes.

### Formula: which one and why

Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined.



$$
\mathrm{score}(c)=P(c)\prod_{w\in V}\left(\frac{c(w,c)+1}{N_c+|V|}\right)^{c(w,d)}
$$



### Step 1. Vocabulary and class priors

V = {amaze, disgust, great, happy, sad, terrible, wonderful}; |V|=7.

| Class | Documents | Prior |
| --- | --- | --- |
| negative | 2 | 1/3 |
| positive | 4 | 2/3 |

### Step 2. Count every training word within each class

| Class | Total tokens | amaze | disgust | great | happy | sad | terrible | wonderful |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| negative | 8 | 0 | 1 | 1 | 1 | 3 | 2 | 0 |
| positive | 13 | 2 | 1 | 4 | 4 | 0 | 0 | 2 |

### Step 3. Count the test words

| Word | Count |
| --- | --- |
| great | 2 |
| happy | 1 |
| disgust | 1 |

OOV words: none

### Step 4. Calculate likelihoods and multiply with the prior

| Class | Prior | Individual factors | Exact score | Decimal score |
| --- | --- | --- | --- | --- |
| negative | 1/3 | great: (1+1)/(8+1×7) = 2/15, used 2 time(s); happy: (1+1)/(8+1×7) = 2/15, used 1 time(s); disgust: (1+1)/(8+1×7) = 2/15, used 1 time(s) | 16/151875 | 0.000105349794239 |
| positive | 2/3 | great: (4+1)/(13+1×7) = 1/4, used 2 time(s); happy: (4+1)/(13+1×7) = 1/4, used 1 time(s); disgust: (1+1)/(13+1×7) = 1/10, used 1 time(s) | 1/960 | 0.00104166666667 |

### Step 5. Compare and optionally normalize

Normalized scores: negative = 0.091846802, positive = 0.908153198

Division by the same total does not change the winning class.

### Final answer

**Predicted class: positive.**

**Yaad rakho:** Count → prior → smoothed word chances → repeated factors → compare.