Natural Language Processing · TEE · papers
Q6a · Henry bigram continuations
2024-12-03 · final · recovered AI paper
Previous examination · complete worked answers
Answer
Given
<s> I am Henry </s>
<s> I like college </s>
<s> Do Henry like college </s>
<s> Henry I am </s>
<s> Do I like Henry </s>
<s> Do I like college </s>
<s> I do like Henry </s>
To find
Most probable next token for each prefix.
Formula: which one and why
LaTeX: P(w\mid h)=\frac{c(h,w)}{\sum_v c(h,v)}Retain case exactly; include sentence-end as an observed next token. Bigram prediction uses only the last token of the prefix.
Step 1. Count each adjacent pair separately within sentences
| Bigram | Count |
|---|---|
| <s> → Do | 3 |
| <s> → Henry | 1 |
| <s> → I | 3 |
| Do → Henry | 1 |
| Do → I | 2 |
| Henry → </s> | 3 |
| Henry → I | 1 |
| Henry → like | 1 |
| I → am | 2 |
| I → do | 1 |
| I → like | 3 |
| am → </s> | 1 |
| am → Henry | 1 |
| college → </s> | 3 |
| do → like | 1 |
| like → Henry | 2 |
| like → college | 3 |
Step 2. Continue <s> Do
| Candidate | Count / history count | Probability |
|---|---|---|
| Henry | 1/3 | 0.333333 |
| I | 2/3 | 0.666667 |
Highest: I.
Step 3. Continue <s> I like
| Candidate | Count / history count | Probability |
|---|---|---|
| Henry | 2/5 | 0.400000 |
| college | 3/5 | 0.600000 |
Highest: college.
Step 4. Continue <s> I
| Candidate | Count / history count | Probability |
|---|---|---|
| am | 2/6 | 0.333333 |
| do | 1/6 | 0.166667 |
| like | 3/6 | 0.500000 |
Highest: like.
Final answer
<s> Do → I
<s> I like → college
<s> I → like
If an examiner explicitly restricts the answer to a lexical word, exclude END and state that restricted candidate set. Do not silently remove an END tie.
Source / convention note. Case is retained as printed: Do and do are distinct. The corpus is not silently grammatically corrected.