Class-style count-table questions and partial-table traps

Question. Train a bigram model on “I am Sam”, “Sam I am”, and “I do not like green eggs and ham”. Include sentence boundaries. Find P(I|<s>), P(Sam|<s>), P(am|I), P(do|I), P(Sam|am) and P(</s>|Sam).

Formula and why it applies

LaTeX: P(w\mid h)=C(h,w)/C(h)

Count all matching histories, including repeated words in different sentence positions.

Solution

Question. A bigram table gives P(i|<s>)=0.25, P(want|i)=0.33, P(english|want)=0.0011, P(food|english)=0.5 and P(</s>|food)=0.68. Score “i want english food” using these supplied rounded values.

Formula and why it applies

LaTeX: P=P(i\mid START)P(want\mid i)P(english\mid want)P(food\mid english)P(END\mid food)

A table-based question uses the provided values; slight differences can arise if exact corpus fractions are used instead.

Solution

Step 1. First two events

LaTeX: 0.25(0.33)=0.0825

Step 2. Add english

LaTeX: 0.0825(0.0011)=0.00009075

Step 3. Add food

LaTeX: 0.00009075(0.5)=0.000045375

Step 4. End the sentence

LaTeX: 0.000045375(0.68)=0.000030855

Question. A corpus has vocabulary size 1446, C(i)=2533 and C(i,want)=827. Compute the unsmoothed and add-one-smoothed P(want|i). The displayed bigram table contains only eight vocabulary columns.

Formula and why it applies

LaTeX: P(want\mid i)=827/2533,\qquad P_L(want\mid i)=\frac{827+1}{2533+1446}

Use the supplied full history total. Summing only the eight displayed columns would wrongly ignore all other possible continuations.

Solution

Step 1. Calculate the unsmoothed ratio

LaTeX: P=827/2533\approx0.326490

Step 2. Add one to the pair count

LaTeX: 827+1=828

Step 3. Add V to the history count

LaTeX: 2533+1446=3979

Step 4. Divide

LaTeX: P_L=828/3979\approx0.208092

Yaad rakho: Partial table ka visible row-sum, full history total nahi hota.