# Q6a · Henry bigram continuations

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f24/q6a

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f24--q6a/

Previous examination · complete worked answers

**Question. From the seven supplied training sentences below, predict the most probable next token for <s> Do ..., <s> I like ..., and <s> I ... using bigrams.** [10 marks]

### Answer

### Given

<s> I am Henry </s>  
<s> I like college </s>  
<s> Do Henry like college </s>  
<s> Henry I am </s>  
<s> Do I like Henry </s>  
<s> Do I like college </s>  
<s> I do like Henry </s>

### To find

Most probable next token for each prefix.

### Formula: which one and why



$$
P(w\mid h)=\frac{c(h,w)}{\sum_v c(h,v)}
$$



Retain case exactly; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix.

### Step 1. Count each adjacent pair separately within sentences

| Bigram | Count |
| --- | --- |
| <s> → Do | 3 |
| <s> → Henry | 1 |
| <s> → I | 3 |
| Do → Henry | 1 |
| Do → I | 2 |
| Henry → </s> | 3 |
| Henry → I | 1 |
| Henry → like | 1 |
| I → am | 2 |
| I → do | 1 |
| I → like | 3 |
| am → </s> | 1 |
| am → Henry | 1 |
| college → </s> | 3 |
| do → like | 1 |
| like → Henry | 2 |
| like → college | 3 |

### Step 2. Continue <s> Do

| Candidate | Count / history count | Probability |
| --- | --- | --- |
| Henry | 1/3 | 0.333333 |
| I | 2/3 | 0.666667 |

Highest: I.

### Step 3. Continue <s> I like

| Candidate | Count / history count | Probability |
| --- | --- | --- |
| Henry | 2/5 | 0.400000 |
| college | 3/5 | 0.600000 |

Highest: college.

### Step 4. Continue <s> I

| Candidate | Count / history count | Probability |
| --- | --- | --- |
| am | 2/6 | 0.333333 |
| do | 1/6 | 0.166667 |
| like | 3/6 | 0.500000 |

Highest: like.

### Final answer

<s> Do → I  
<s> I like → college  
<s> I → like

If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie.

**Source / convention note.** Case is retained as printed: Do and do are distinct. The corpus is not silently grammatically corrected.