Natural Language Processing · M2 · mocks

Q3 (3 marks)

M2 Mock A · core calculations

M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice

Q3

Training corpus: “I like tea” and “I like coffee”. Lowercase all words and add <s> and </s> to each sentence. Using add-one bigram smoothing and a vocabulary containing both boundary symbols, find P(“i like tea”). (3 marks)

Given

Padded corpus: <s> i like tea </s>; <s> i like coffee </s>. V={<s>, i, like, tea, coffee, </s>}, |V|=6.

To find

Product of the four smoothed transition probabilities, including the end transition.

Formula and why it applies

P_{+1}(w\mid h)=\frac{C(hw)+1}{C(h)+|V|}

Every possible vocabulary continuation gets one pseudocount. Both boundaries are counted in V because the question explicitly requires the course convention.

Solution

Step 1. Count histories and pairs

Transition	Pair count	History count
<s> → i	2	2
i → like	2	2
like → tea	1	2
tea → </s>	1	1

Step 2. Compute first probability

P(i\mid\langle s\rangle)=\frac{2+1}{2+6}=\frac38

Step 3. Compute middle probabilities

P(\text{like}\mid i)=\frac38

P(\text{tea}\mid\text{like})=\frac{1+1}{2+6}=\frac14

Step 4. Compute end probability

P(\langle/ s\rangle\mid\text{tea})=\frac{1+1}{1+6}=\frac27

Step 5. Multiply

P=\frac38\times\frac38\times\frac14\times\frac27=\frac9{896}\approx0.010044643

Sentence probability = 9/896 ≈ 0.010044643.