Natural Language Processing · TEE · papers

Q5B · Naïve Bayes word-sense classification

Solved final paper · December 2025

Actual final examination · 2 December 2025 · 100 marks · 3 hours

Question. Consider the sentence: “The bat hit the ball hard on the cricket field.” Two possible senses of “bat”: Sense 1: Bat (Sports equipment – cricket/baseball); Sense 2: Bat (Nocturnal flying mammal).

Sentence ID	Context words	Sense
1	ball, cricket, score, stadium	Sports
2	night, cave, wings, fly	Animal
3	player, run, match, bat	Sports
4	nocturnal, vampire, insect	Animal
5	team, field, bowl, bat	Sports
6	dark, flying, cave, night	Animal

a) Using the Naïve Bayes algorithm, classify the sense of “bat” in the given sentence. Apply Laplace Smoothing wherever applicable.
b) Discuss the significance of Laplace Smoothing in Naïve Bayes Classification. [10 marks]

Answer

Given

There are three Sports examples and three Animal examples. Treat training context lists as bag-of-words token counts. Exclude the ambiguous target “bat” from the test context. Use the fixed training vocabulary; ignore out-of-vocabulary test words hit and hard. The usable test features are ball, cricket and field.

To find

The posterior-score comparison for Sports and Animal, and the role of smoothing.

Formula: which one and why

\mathrm{score}(s)=P(s)\prod_{w\in\mathrm{context}}P(w\mid s)

P(w\mid s)=\frac{C(w,s)+1}{N_s+V}

P(s) is the class prior, C(w,s) counts occurrences of context word w in class s, Nₛ is the total number of context tokens in that class and V is the shared vocabulary size. Conditional independence lets us multiply word probabilities. The common denominator P(context) is unnecessary when only comparing classes.

Solution

Step 1. Calculate the priors

P(S)=\frac36=0.5,\qquad P(A)=\frac36=0.5

Step 2. Count class tokens and vocabulary

Sports has 4 + 4 + 4 = 12 tokens. Animal has 4 + 3 + 4 = 11 tokens. The shared training vocabulary has 20 distinct words: ball, cricket, score, stadium, player, run, match, bat, team, field, bowl, night, cave, wings, fly, nocturnal, vampire, insect, dark, flying. “bat”, “night” and “cave” repeat but count only once each towards V.

Step 3. Look up each usable context word

Feature	Sports count	P(word \| Sports)	P(word \| Animal)
ball	1	(1+1)/(12+20) = 2/32	(0+1)/(11+20) = 1/31
cricket	1	2/32	1/31
field	1	2/32	1/31

Step 4. Multiply the Sports score

\mathrm{score}(S)=0.5\times\frac2{32}\times\frac2{32}\times\frac2{32}=0.0001220703125

Step 5. Multiply the Animal score

\mathrm{score}(A)=0.5\times\frac1{31}\times\frac1{31}\times\frac1{31}\approx0.00001678359236

Step 6. Compare the scores

0.0001220703125 is greater than 0.00001678359236. Therefore select Sports. Optional normalization gives P(Sports | retained features) ≈ 0.87913; normalization is not needed for the decision.

Final answer

The intended sense is bat as sports equipment. Ball, cricket and field favour that sense.

Yaad rakho: Pehle class prior, phir har context-word likelihood multiply; bigger score wali sense choose karo.

Why Laplace smoothing matters

Without smoothing, all three test features have zero count in Animal. One zero factor makes the entire class score zero, even if other evidence could support that class. Add-one smoothing assigns nonzero probability to every word in the defined vocabulary and redistributes probability mass. It does not mean every unseen word outside the vocabulary can be inserted without changing the vocabulary model.

If following the supplied paper key exactly

The key also multiplies probabilities for hit and hard using the same denominators 32 and 31. That convention produces Sports = 1.19209290 × 10⁻⁷ and Animal ≈ 1.74647163 × 10⁻⁸, again selecting Sports. Its treatment of out-of-vocabulary words is not a normalized fixed-vocabulary model. State your feature convention clearly; the complete calculation above uses a consistent fixed training vocabulary.