Natural Language Processing · TEE · papers
Q7b · Naive Bayes sentiment
2024-02-12 · re-exam · recovered AI paper
Previous examination · complete worked answers
Question. Train Naive Bayes on: positive “amaze wonderful great”; “happy happy great”; “amaze great happy disgust”; “great happy wonderful”. Negative “disgust sad terrible great”; “sad terrible sad happy”. Predict sentence 7: “great happy disgust great”. [10 marks]
Answer
Given
| Training text | Class |
|---|---|
| amaze wonderful great | positive |
| happy happy great | positive |
| amaze great happy disgust | positive |
| great happy wonderful | positive |
| disgust sad terrible great | negative |
| sad terrible sad happy | negative |
Test: great happy disgust great
To find
Classify the test using multinomial Naive Bayes.
Formula: which one and why
Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined.
LaTeX: \mathrm{score}(c)=P(c)\prod_{w\in V}\left(\frac{c(w,c)+1}{N_c+|V|}\right)^{c(w,d)}Step 1. Vocabulary and class priors
V = {amaze, disgust, great, happy, sad, terrible, wonderful}; |V|=7.
| Class | Documents | Prior |
|---|---|---|
| negative | 2 | 1/3 |
| positive | 4 | 2/3 |
Step 2. Count every training word within each class
| Class | Total tokens | amaze | disgust | great | happy | sad | terrible | wonderful |
|---|---|---|---|---|---|---|---|---|
| negative | 8 | 0 | 1 | 1 | 1 | 3 | 2 | 0 |
| positive | 13 | 2 | 1 | 4 | 4 | 0 | 0 | 2 |
Step 3. Count the test words
| Word | Count |
|---|---|
| great | 2 |
| happy | 1 |
| disgust | 1 |
OOV words: none
Step 4. Calculate likelihoods and multiply with the prior
| Class | Prior | Individual factors | Exact score | Decimal score |
|---|---|---|---|---|
| negative | 1/3 | great: (1+1)/(8+1×7) = 2/15, used 2 time(s); happy: (1+1)/(8+1×7) = 2/15, used 1 time(s); disgust: (1+1)/(8+1×7) = 2/15, used 1 time(s) | 16/151875 | 0.000105349794239 |
| positive | 2/3 | great: (4+1)/(13+1×7) = 1/4, used 2 time(s); happy: (4+1)/(13+1×7) = 1/4, used 1 time(s); disgust: (1+1)/(13+1×7) = 1/10, used 1 time(s) | 1/960 | 0.00104166666667 |
Step 5. Compare and optionally normalize
Normalized scores: negative = 0.091846802, positive = 0.908153198
Division by the same total does not change the winning class.
Final answer
Predicted class: positive.
Yaad rakho: Count → prior → smoothed word chances → repeated factors → compare.