Natural Language Processing · TEE · papers
Q4c · Most probable bigram continuations
2025-02-21 · re-exam · recovered AI paper
Previous examination · complete worked answers
Answer
Given
<s> I love programming </s>
<s> Programming is fun </s>
<s> I love coding </s>
<s> Coding is great </s>
To find
Most probable next token for each prefix.
Formula: which one and why
LaTeX: P(w\mid h)=\frac{c(h,w)}{\sum_v c(h,v)}Lowercase lexical tokens; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix.
Step 1. Count each adjacent pair separately within sentences
| Bigram | Count |
|---|---|
| <s> → coding | 1 |
| <s> → i | 2 |
| <s> → programming | 1 |
| coding → </s> | 1 |
| coding → is | 1 |
| fun → </s> | 1 |
| great → </s> | 1 |
| i → love | 2 |
| is → fun | 1 |
| is → great | 1 |
| love → coding | 1 |
| love → programming | 1 |
| programming → </s> | 1 |
| programming → is | 1 |
Step 2. Continue <s> I love
| Candidate | Count / history count | Probability |
|---|---|---|
| coding | 1/2 | 0.500000 |
| programming | 1/2 | 0.500000 |
Highest: programming, coding. This is a tie; no tie-break rule is supplied.
Step 3. Continue <s> programming
| Candidate | Count / history count | Probability |
|---|---|---|
| </s> | 1/2 | 0.500000 |
| is | 1/2 | 0.500000 |
Highest: </s>, is. This is a tie; no tie-break rule is supplied.
Step 4. Continue <s> coding
| Candidate | Count / history count | Probability |
|---|---|---|
| </s> | 1/2 | 0.500000 |
| is | 1/2 | 0.500000 |
Highest: </s>, is. This is a tie; no tie-break rule is supplied.
Final answer
<s> I love → programming or coding
<s> programming → </s> or is
<s> coding → </s> or is
If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie.