Intelligent Systems II · TEE · papers

Q4(a) Naive Bayes and Laplace smoothing [10]

Re-examination · 25 Jun 2024

Actual historical paper ·100 marks attempted /140 offered · Q1 compulsory + any four of Q2–Q7. Source defects and solution assumptions are explicitly identified.

Question. Q4(a) [10 marks · 2024-06-25]

(1) Use Naive Bayes on the dataset to predict whether a person buys a computer for Age=Old, Income=Medium, Freelancer=Yes. (2) Explain Laplacian smoothing, whether it is required here and its impact.

ID	Age	Income	Freelancer	Buys computer
1	Young	High	No	No
2	Young	High	No	No
3	Middle Age	High	No	Yes
4	Old	Medium	No	Yes
5	Old	Low	Yes	Yes
6	Old	Low	Yes	No
7	Middle Age	Low	Yes	Yes
8	Young	Medium	No	No
9	Young	Low	Yes	Yes
10	Young	Medium	Yes	Yes

Answer

Given

Ten records; query (Old, Medium, Yes). Class counts Yes 6, No 4.

To find

Class scores, normalized comparison and smoothing impact.

Formula and why it applies

Naive Bayes multiplies the class prior and feature likelihoods, assuming features are conditionally independent given the class. The same evidence denominator cancels when comparing classes.

s(c)=P(c)\prod_j P(x_j\mid c),\qquad P(c\mid x)=\frac{s(c)}{\sum_{k}s(k)}

Solution

Step 1. Count query-feature matches within each class

Class	N	Old	Medium income	FreelancerYes
Yes	6	2	2	4
No	4	1	1	1

Yes records are 3,4,5,7,9,10; No records 1,2,6,8. Count within the chosen class, not among all ten records.

Step 2. Compute unsmoothed Yes score

s(Y)=\frac6{10}\frac26\frac26\frac46=\frac2{45}\approx0.0444444

Step 3. Compute unsmoothed No score

s(N)=\frac4{10}\frac14\frac14\frac14=\frac1{160}=0.00625

Step 4. Normalize and classify

P(Y\mid x)=\frac{2/45}{2/45+1/160}=\frac{64}{73}\approx0.876712

P(N\mid x)=\frac9{73}\approx0.123288

Predict Yes because its score is larger.

Step 5. Explain add-one smoothing

A zero likelihood would make the whole class product zero, even if other features favor the class. Add-one smoothing prevents that by adding one to each category count and adding the number of possible categories to the denominator. No query likelihood is zero here, so smoothing is not needed to rescue a zero score; it still changes the estimates.

P(x_j=v\mid c)=\frac{N_{jvc}+1}{N_c+K_j}

There are 3 age categories,3 income categories and 2 freelancer categories. Retaining empirical priors:

s_{+1}(Y)=\frac6{10}\frac{2+1}{6+3}\frac{2+1}{6+3}\frac{4+1}{6+2}=\frac1{24}\approx0.0416667

s_{+1}(N)=\frac4{10}\frac{1+1}{4+3}\frac{1+1}{4+3}\frac{1+1}{4+2}=\frac8{735}\approx0.0108844

Step 6. Compare the smoothed result

P_{+1}(Y\mid x)=\frac{1/24}{1/24+8/735}=\frac{245}{309}\approx0.792880

The predicted class remains Yes, while certainty decreases. Stating whether priors are smoothed matters; here only feature likelihoods were add-one smoothed.

Final answer. Unsmoothed predictionYes, posterior 64/73. Add-one feature smoothing still predictsYes, posterior 245/309.