# A complete unseen-bigram question from corpus to final answer

Subject: Natural Language Processing

Exam: M2

Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/f-smoothing/smoothed-sentence

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/f-smoothing--smoothed-sentence/

Formulas explained through full worked examples

**Question. Consider the corpus “I love to code in Python. I love to learn new things. I code every day.” Obtain the bigram probabilities needed for “I love Python”. Explain the zero-probability issue, then use Laplace smoothing to score it. Treat each sentence separately, preserve case, remove punctuation, and include an end token among predicted outcomes.**

#### Given

Three sentences padded with one start/end marker. Distinct words: I, love, to, code, in, Python, learn, new, things, every, day. With </s>, V=12. Start is context only.

#### To find

Unsmoothed and smoothed P(I love Python </s> | <s>).

#### Formula and why it applies



$$
P(w\mid h)=\frac{C(h,w)}{C(h)},\qquad P_L(w\mid h)=\frac{C(h,w)+1}{C(h)+12}
$$



The test contains an unseen transition love→Python. Smoothing assigns probability to that transition and renormalizes every required history row.

#### Solution

##### Step 1. Count the contexts

| History | Count |
| --- | --- |
| <s> | 3 |
| I | 3 |
| love | 2 |
| Python | 1 |

##### Step 2. Count each required pair

| Pair | Count |
| --- | --- |
| <s> I | 3 |
| I love | 2 |
| love Python | 0 |
| Python </s> | 1 |

##### Step 3. Compute the unsmoothed product



$$
P=\frac33\times\frac23\times\frac02\times\frac11=0
$$



Both words love and Python were seen, but their adjacent pair was not. This is an unseen bigram, not an unknown-word case.

##### Step 4. Smooth the first event



$$
P_L(I\mid\langle s\rangle)=\frac{3+1}{3+12}=\frac4{15}
$$



##### Step 5. Smooth the second event



$$
P_L(love\mid I)=\frac{2+1}{3+12}=\frac3{15}
$$



##### Step 6. Smooth the unseen event



$$
P_L(Python\mid love)=\frac{0+1}{2+12}=\frac1{14}
$$



##### Step 7. Smooth the end event



$$
P_L(\langle/s\rangle\mid Python)=\frac{1+1}{1+12}=\frac2{13}
$$



##### Step 8. Multiply the smoothed factors



$$
P_L=\frac4{15}\times\frac3{15}\times\frac1{14}\times\frac2{13}
$$





$$
P_L=\frac{24}{40950}=\frac4{6825}\approx0.000586081
$$



**Final answer.** Unsmoothed P=0. With the stated V=12 and boundaries, Laplace-smoothed P≈0.000586081.

If an exam states a different convention—for example excluding boundary tokens from the entire calculation—recompute both V and the set of probability factors consistently. Do not combine a boundary-aware numerator with a boundary-free denominator.