# Q5b · Multinomial Naive Bayes

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q5b

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q5b/

Previous examination · complete worked answers

**Question. Explain Naive Bayes for text classification. Using the four documents below, classify “Chinese Chinese Chinese Tokyo Japan”: Chinese Beijing Chinese=yes; Chinese Chinese Shanghai=yes; Chinese Macao=yes; Tokyo Japan Chinese=no.** [10 marks]

### Answer

Multinomial NB assumes token draws are conditionally independent given the class; it combines the class prior and class-specific word probabilities. Repeated Chinese contributes its likelihood three times.

### Given

| Training text | Class |
| --- | --- |
| Chinese Beijing Chinese | yes |
| Chinese Chinese Shanghai | yes |
| Chinese Macao | yes |
| Tokyo Japan Chinese | no |

Test: Chinese Chinese Chinese Tokyo Japan

### To find

Classify the test using multinomial Naive Bayes.

### Formula: which one and why

Assume add-one smoothing unless the question supplies probabilities. Keep repeated words as repeated factors. Use training vocabulary only; ignore out-of-vocabulary words unless an unknown-token model is defined.



$$
\mathrm{score}(c)=P(c)\prod_{w\in V}\left(\frac{c(w,c)+1}{N_c+|V|}\right)^{c(w,d)}
$$



### Step 1. Vocabulary and class priors

V = {beijing, chinese, japan, macao, shanghai, tokyo}; |V|=6.

| Class | Documents | Prior |
| --- | --- | --- |
| no | 1 | 1/4 |
| yes | 3 | 3/4 |

### Step 2. Count every training word within each class

| Class | Total tokens | beijing | chinese | japan | macao | shanghai | tokyo |
| --- | --- | --- | --- | --- | --- | --- | --- |
| no | 3 | 0 | 1 | 1 | 0 | 0 | 1 |
| yes | 8 | 1 | 5 | 0 | 1 | 1 | 0 |

### Step 3. Count the test words

| Word | Count |
| --- | --- |
| chinese | 3 |
| tokyo | 1 |
| japan | 1 |

OOV words: none

### Step 4. Calculate likelihoods and multiply with the prior

| Class | Prior | Individual factors | Exact score | Decimal score |
| --- | --- | --- | --- | --- |
| no | 1/4 | chinese: (1+1)/(3+1×6) = 2/9, used 3 time(s); tokyo: (1+1)/(3+1×6) = 2/9, used 1 time(s); japan: (1+1)/(3+1×6) = 2/9, used 1 time(s) | 8/59049 | 0.000135480702467 |
| yes | 3/4 | chinese: (5+1)/(8+1×6) = 3/7, used 3 time(s); tokyo: (0+1)/(8+1×6) = 1/14, used 1 time(s); japan: (0+1)/(8+1×6) = 1/14, used 1 time(s) | 81/268912 | 0.000301213779973 |

### Step 5. Compare and optionally normalize

Normalized scores: no = 0.310241388, yes = 0.689758612

Division by the same total does not change the winning class.

### Final answer

**Predicted class: yes.**

**Yaad rakho:** Count → prior → smoothed word chances → repeated factors → compare.