# Q4c · Most probable bigram continuations

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q4c

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4c/

Previous examination · complete worked answers

**Question. Train on <s> I love programming </s>; <s> Programming is fun </s>; <s> I love coding </s>; <s> Coding is great </s>. Predict next tokens for <s> I love ..., <s> programming ..., <s> coding ... using a bigram model.** [10 marks]

### Answer

### Given

<s> I love programming </s>  
<s> Programming is fun </s>  
<s> I love coding </s>  
<s> Coding is great </s>

### To find

Most probable next token for each prefix.

### Formula: which one and why



$$
P(w\mid h)=\frac{c(h,w)}{\sum_v c(h,v)}
$$



Lowercase lexical tokens; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix.

### Step 1. Count each adjacent pair separately within sentences

| Bigram | Count |
| --- | --- |
| <s> → coding | 1 |
| <s> → i | 2 |
| <s> → programming | 1 |
| coding → </s> | 1 |
| coding → is | 1 |
| fun → </s> | 1 |
| great → </s> | 1 |
| i → love | 2 |
| is → fun | 1 |
| is → great | 1 |
| love → coding | 1 |
| love → programming | 1 |
| programming → </s> | 1 |
| programming → is | 1 |

### Step 2. Continue <s> I love

| Candidate | Count / history count | Probability |
| --- | --- | --- |
| coding | 1/2 | 0.500000 |
| programming | 1/2 | 0.500000 |

Highest: programming, coding. This is a tie; no tie-break rule is supplied.

### Step 3. Continue <s> programming

| Candidate | Count / history count | Probability |
| --- | --- | --- |
| </s> | 1/2 | 0.500000 |
| is | 1/2 | 0.500000 |

Highest: </s>, is. This is a tie; no tie-break rule is supplied.

### Step 4. Continue <s> coding

| Candidate | Count / history count | Probability |
| --- | --- | --- |
| </s> | 1/2 | 0.500000 |
| is | 1/2 | 0.500000 |

Highest: </s>, is. This is a tie; no tie-break rule is supplied.

### Final answer

<s> I love → programming or coding  
<s> programming → </s> or is  
<s> coding → </s> or is

If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie.