Natural Language Processing · M2 · formulas

Mam's bank numerical—every count and every multiplication

WSD: Lesk & Naive Bayes

Formulas explained through full worked examples

Question. Classify the sense of “bank” using the training table below. The sentence is: “As the stream flowed gently past the shore, we set up camp by the bank of the river, discussing the possibility of borrowing a loan to buy a boat for future trips.” Use only the highlighted feature words stream, shore and loan, and add-one multinomial smoothing.

Given

Training row	Context words	Sense
1	money, deposit, loan	Finance
2	river, water, shore	Riverbank
3	account, savings, investment	Finance
4	boat, river, stream	Riverbank
5	branch, deposit, check	Finance
6	fishing, shore, river	Riverbank

Query feature set: {stream, shore, loan}. Other sentence words are not included as features in this supplied example.

To find

The most probable bank sense and normalized posteriors.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\operatorname{score}(s)=P(s)\prod_j P(w_j\mid s).

For add-one multinomial smoothing:

P(w\mid s)=\frac{C(w,s)+1}{N_s+V}.

$C(w,s)$ counts occurrences of word $w$ in training contexts of sense $s$ ; $N_s$ is the total number of word occurrences in those contexts; $V$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Calculate priors

Finance appears in 3 of the 6 rows; Riverbank appears in 3 of 6.

P(F)=\frac36=\frac12,\qquad P(R)=\frac36=\frac12.

Step 2. Construct the vocabulary

{money, deposit, loan, river, water, shore, account, savings, investment, boat, stream, fishing, branch, check}. Thus V=14. Repeated deposit or river is counted once in vocabulary.

Step 3. Count word occurrences per class

There are 3 finance rows × 3 words = 9 word occurrences. There are 3 river rows × 3 words = 9. Thus $N_F=N_R=9$ , and each smoothed denominator is $9+14=23$ .

Step 4. Count the three query words

Word	Finance count	Riverbank count
stream	0	1
shore	0	2
loan	1	0

Step 5. Calculate finance likelihoods

P(\text{stream}\mid F)=\frac{0+1}{23}=\frac1{23}

P(\text{shore}\mid F)=\frac{0+1}{23}=\frac1{23}

P(\text{loan}\mid F)=\frac{1+1}{23}=\frac2{23}

Step 6. Multiply the finance score

\operatorname{score}(F)=\frac12\times\frac1{23}\times\frac1{23}\times\frac2{23}=\frac1{23^3}=\frac1{12167}\approx0.00008219.

Step 7. Calculate river likelihoods

P(\text{stream}\mid R)=\frac{1+1}{23}=\frac2{23}

P(\text{shore}\mid R)=\frac{2+1}{23}=\frac3{23}

P(\text{loan}\mid R)=\frac{0+1}{23}=\frac1{23}

Step 8. Multiply the river score

\operatorname{score}(R)=\frac12\times\frac2{23}\times\frac3{23}\times\frac1{23}=\frac3{12167}\approx0.00024657.

Step 9. Compare scores

3/12167 is larger than 1/12167. Riverbank wins even though “loan” is financial evidence: the combined clues matter.

Step 10. Normalize if a posterior is requested

P(F\mid C)=\frac{1/12167}{4/12167}=\frac14=0.25

P(R\mid C)=\frac{3/12167}{4/12167}=\frac34=0.75.

Answer. Bank is the riverbank sense. Scores are 1/12167 and 3/12167; normalized model posteriors are 0.25 and 0.75.

Without smoothing: stream and shore are absent in Finance; loan is absent in Riverbank. Both unsmoothed scores become zero, so there is no valid winning class from those products. Also, unnormalized scores should not be labelled as final posterior probabilities.

Zero word? Add-one: count+1 upar, total words+vocabulary neeche. Last mein score bada wala sense.