# Q3 (3 marks)

Subject: Natural Language Processing

Exam: M2

Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-c/q3

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-c--q3/

M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice

### Q3

For trigram prediction, C(i like)=5, C(i like tea)=3 and C(i like coffee)=2. Find the MLE probability of each continuation. Then find add-one P(tea|i like) using a continuation vocabulary of size 4. (3 marks)

**Given**

Two-word history i like occurs 5 times; continuations tea=3, coffee=2; |V|=4.

**To find**

Two MLE values and the smoothed tea probability.

### Formula and why it applies



$$
P(w_i\mid w_{i-2},w_{i-1})=\frac{C(w_{i-2},w_{i-1},w_i)}{C(w_{i-2},w_{i-1})}
$$





$$
P_{+1}(w\mid h)=\frac{C(hw)+1}{C(h)+|V|}
$$



A trigram conditions on two preceding words. The same add-one count rule applies to that longer history.

### Solution

#### Step 1. Tea MLE



$$
P(\text{tea}\mid i,\text{like})=\frac35=0.6
$$



#### Step 2. Coffee MLE



$$
P(\text{coffee}\mid i,\text{like})=\frac25=0.4
$$



#### Step 3. Smooth tea



$$
P_{+1}(\text{tea}\mid i,\text{like})=\frac{3+1}{5+4}=\frac49\approx0.444444
$$



MLE tea=0.6; coffee=0.4; smoothed tea=4/9≈0.444444.