# Q1a · Normalization changes vocabulary

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r23/q1a

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r23--q1a/

Previous examination · complete worked answers

**Question. “The vocabulary of a corpus remains the same before and after normalization.” State true or false and justify with an example.** [5 marks]

### Answer

**False in general.** Vocabulary is the set of distinct tokens under a chosen tokenizer. In “Cats cats running”, raw case-sensitive vocabulary={Cats,cats,running}, size3. Lowercase merges Cats/cats: {cats,running}, size2. Lemmatization may yield {cat,run}; stopword removal can remove entries. For an already normalized corpus the set may happen to remain unchanged, so the correct claim is that normalization can change it, not that it always must.