Natural Language Processing · M2 · formulas

More WSD numericals: given likelihoods, unequal priors and repeated clues

WSD: Lesk & Naive Bayes

Formulas explained through full worked examples

Question. For bass, let P(Fish)=0.6 and P(Music)=0.4. Given Fish likelihoods P(caught|Fish)=0.5, P(lake|Fish)=0.4 and Music likelihoods P(caught|Music)=0.05, P(lake|Music)=0.1, classify context {caught,lake} and normalize.

Given

Priors and likelihoods are provided directly; no smoothing or training-count estimation is requested.

To find

The winning sense and its posterior.

Formula and why it applies

$\operatorname{score}(s)=P(s)P(\text{caught}\mid s)P(\text{lake}\mid s)$ , because the two features are treated as conditionally independent.

Step 1. Fish multiplication

$0.6\times0.5=0.3$ , then $0.3\times0.4=0.12$ .

Step 2. Music multiplication

$0.4\times0.05=0.02$ , then $0.02\times0.1=0.002$ .

Step 3. Total evidence

$0.12+0.002=0.122$ .

Step 4. Normalize

P(Fish\mid C)=0.12/0.122\approx0.983607

P(Music\mid C)=0.002/0.122\approx0.016393.

Answer. Fish, with model posterior approximately 0.983607.

Question. Disambiguate bank from the repeated context tokens {loan, loan, river}. Priors are Finance=2/3 and River=1/3. Vocabulary V=4. Finance has N=8 total tokens, loan count 3 and river count 0; River has N=4, loan count 0 and river count 2. Apply add-one smoothing.

Given

Multinomial counts are supplied. The two appearances of loan must both contribute.

To find

Scores and normalized Finance posterior.

Formula and why it applies

Naive Bayes selects the sense with the largest prior × likelihood product. The context evidence denominator is identical across candidate senses, so it can be omitted for comparison.

\operatorname{score}(s)=P(s)\prod_j P(w_j\mid s).

For add-one multinomial smoothing:

P(w\mid s)=\frac{C(w,s)+1}{N_s+V}.

$C(w,s)$ counts occurrences of word $w$ in training contexts of sense $s$ ; $N_s$ is the total number of word occurrences in those contexts; $V$ is the shared vocabulary size. It is not the number of training sentences. Repeated query words contribute repeated factors.

Step 1. Finance denominator

$8+4=12$ . $P(loan\mid F)=(3+1)/12=1/3$ ; $P(river\mid F)=(0+1)/12=1/12$ .

Step 2. River denominator

$4+4=8$ . $P(loan\mid R)=(0+1)/8=1/8$ ; $P(river\mid R)=(2+1)/8=3/8$ .

Step 3. Finance product

\operatorname{score}(F)=\frac23\times\frac13\times\frac13\times\frac1{12}=\frac1{162}\approx0.00617284.

Step 4. River product

\operatorname{score}(R)=\frac13\times\frac18\times\frac18\times\frac38=\frac1{512}\approx0.001953125.

Step 5. Compare and normalize

P(F\mid C)=\frac{1/162}{1/162+1/512}=\frac{512}{674}=\frac{256}{337}\approx0.759644.

Answer. Finance wins. Repeated loan contributes a squared likelihood; do not reduce the query to a unique-word set.

Answer-writing order in an exam

Write the class priors, vocabulary size, class token totals, a likelihood table, both score products and the final class. Keep fractions until the comparison where possible. If the question only asks classification, normalization is optional; if it asks posterior probabilities, normalization is required.

Repeated word aaye toh factor bhi repeat. Sirf word-set bana diya toh multinomial answer badal jayega.