Natural Language Processing · M2 · mocks

Q1 (2 marks)

M2 Mock B · unseen pairs & decoding

M2 · confirmed 2 + 2 + any two of 3 + 3 + 3 · worked practice

Q1

A language model gives a test sequence probability of 0.0016 across four predicted tokens. Calculate perplexity. (2 marks)

Given

P=0.0016; N=4.

To find

PP.

Formula and why it applies

PP=P^{-1/N}=\left(\frac1P\right)^{1/N}LaTeX: PP=P^{-1/N}=\left(\frac1P\right)^{1/N}

Use the number of predicted events, not an unrelated training token count.

Solution

Step 1. Reciprocal

\frac1{0.0016}=625LaTeX: \frac1{0.0016}=625

Step 2. Fourth root

PP=625^{1/4}=5LaTeX: PP=625^{1/4}=5
Perplexity=5.