Natural Language Processing · M2 · read

Bayes’ rule: prior, evidence and answer

Start here · language & probability

Prerequisites for M2 · no earlier NLP knowledge assumed

Bayes’ rule reverses a conditional probability. A word-sense classifier asks “which sense is most likely given these context words?” Training data more easily supplies “how likely are these words when this sense is used?” Bayes connects the two.

LaTeX: P(c\mid x)=\frac{P(x\mid c)P(c)}{P(x)}

For ranking classes, calculate prior × likelihood. The common denominator does not affect which class is largest. For an actual posterior probability, divide by the sum of all class scores.

A complete Bayes calculation

A word has two senses: financial bank and river bank. Priors are 0.6 and 0.4. The probability of observing “loan” is 0.5 under the financial sense and 0.1 under the river sense. Find the most likely sense and its posterior.

Given

P(F) = 0.6; P(R) = 0.4; P(loan | F) = 0.5; P(loan | R) = 0.1. These two senses are exhaustive.

To find

Scores and P(F | loan).

Formula and why it applies

LaTeX: s(c)=P(c)P(\text{loan}\mid c)

LaTeX: P(F\mid\text{loan})=\frac{s(F)}{s(F)+s(R)}

Multiply prior and likelihood to reverse the conditioning. Then normalise because a posterior was requested.

Solution

Step 1. Score the financial sense

LaTeX: s(F)=0.6\times0.5=0.30

Step 2. Score the river sense

LaTeX: s(R)=0.4\times0.1=0.04

Step 3. Add the scores

LaTeX: P(\text{loan})=0.30+0.04=0.34

Step 4. Normalise the financial score

LaTeX: P(F\mid\text{loan})=\frac{0.30}{0.34}=\frac{15}{17}\approx0.882353

Financial bank wins; its posterior probability is about 0.882353.

Yaad rakho: Pehle se kitna common × evidence kitna suit karta = score.