Character-model perplexity: keep exact probabilities

Question. A unigram character model gives P(A)=0.25, P(B)=0.50, P(C)=0.25. Find the perplexity of AAA and ABC. Then find both perplexities when the three characters are equally likely.

Formula and why it applies

LaTeX: P(c_1c_2c_3)=P(c_1)P(c_2)P(c_3),\qquad PP=P(W)^{-1/3}

Characters, not words, are the model's prediction units in this question. Use exact 1/3 for uniform probabilities, rather than rounded 0.33.

Solution

Step 1. Score AAA

LaTeX: P(AAA)=0.25^3=\frac1{64}=0.015625

Step 2. Find AAA perplexity

LaTeX: PP(AAA)=64^{1/3}=4

Step 3. Score ABC

LaTeX: P(ABC)=0.25(0.5)(0.25)=\frac1{32}=0.03125

Step 4. Find ABC perplexity

LaTeX: PP(ABC)=32^{1/3}\approx3.174802

Step 5. Change to an exactly uniform model

LaTeX: P(AAA)=P(ABC)=(1/3)^3=1/27

Step 6. Find both uniform-model perplexities

LaTeX: PP=27^{1/3}=3