# Q3 (3 marks)

Subject: Natural Language Processing

Exam: M2

Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/mocks/mock-a/q3

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/mocks/mock-a--q3/

M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice

### Q3

Training corpus: “I like tea” and “I like coffee”. Lowercase all words and add <s> and </s> to each sentence. Using add-one bigram smoothing and a vocabulary containing both boundary symbols, find P(“i like tea”). (3 marks)

**Given**

Padded corpus: <s> i like tea </s>; <s> i like coffee </s>. V={<s>, i, like, tea, coffee, </s>}, |V|=6.

**To find**

Product of the four smoothed transition probabilities, including the end transition.

### Formula and why it applies



$$
P_{+1}(w\mid h)=\frac{C(hw)+1}{C(h)+|V|}
$$



Every possible vocabulary continuation gets one pseudocount. Both boundaries are counted in V because the question explicitly requires the course convention.

### Solution

#### Step 1. Count histories and pairs

| Transition | Pair count | History count |
| --- | --- | --- |
| <s> → i | 2 | 2 |
| i → like | 2 | 2 |
| like → tea | 1 | 2 |
| tea → </s> | 1 | 1 |

#### Step 2. Compute first probability



$$
P(i\mid\langle s\rangle)=\frac{2+1}{2+6}=\frac38
$$



#### Step 3. Compute middle probabilities



$$
P(\text{like}\mid i)=\frac38
$$





$$
P(\text{tea}\mid\text{like})=\frac{1+1}{2+6}=\frac14
$$



#### Step 4. Compute end probability



$$
P(\langle/ s\rangle\mid\text{tea})=\frac{1+1}{1+6}=\frac27
$$



#### Step 5. Multiply



$$
P=\frac38\times\frac38\times\frac14\times\frac27=\frac9{896}\approx0.010044643
$$



Sentence probability = 9/896 ≈ 0.010044643.