Natural Language Processing · TEE · read-chapter

Perplexity, log probabilities and smoothing

Unit 3, pages 121–156

1. Perplexity: meaning, formula and four complete examples
2. Log probabilities: replace long products with sums
3. Laplace smoothing: why both numerator and denominator change
4. A complete unseen-bigram question from corpus to final answer
5. Adjusted counts and relative discounting
6. Character-model perplexity: keep exact probabilities