# Q3 (3 marks)

Subject: Natural Language Processing

Exam: M2

Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-b/q3

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-b--q3/

M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice

### Q3

For a fixed vocabulary of five possible continuations, C(very)=7 and C(very bright)=0. Calculate the unsmoothed and Laplace-smoothed probabilities of bright after very. (3 marks)

**Given**

History count=7; pair count=0; |V|=5.

**To find**

MLE and add-one probabilities.

### Formula and why it applies



$$
P_{MLE}(w\mid h)=\frac{C(hw)}{C(h)}
$$





$$
P_{+1}(w\mid h)=\frac{C(hw)+1}{C(h)+|V|}
$$



MLE uses only observations. Laplace reserves probability for all known vocabulary outcomes, including unseen pairs.

### Solution

#### Step 1. Unsmoothed estimate



$$
P_{MLE}(\text{bright}\mid\text{very})=\frac07=0
$$



#### Step 2. Add one to the pair count



$$
0+1=1
$$



#### Step 3. Add vocabulary size to the history count



$$
7+5=12
$$



#### Step 4. Divide



$$
P_{+1}(\text{bright}\mid\text{very})=\frac1{12}\approx0.083333
$$



MLE=0; Laplace=1/12≈0.083333. The zero count means unseen in this corpus, not impossible language.