# Q5a · Preprocess the NLP passage

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q5a

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q5a/

Previous examination · complete worked answers

**Question. Perform the requested text preprocessing on: “Natural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between computers and humans through natural language. The ultimate objective of NLP is to read, decipher, understand, and make sense of human languages in a valuable way.” Remove the supplied stopwords {is,a,of,that,on,I,the,and,to,in}.** [6 marks]

### Answer

| Stage | Result |
| --- | --- |
| Sentence segmentation | Natural Language Processing (NLP) is a field of artificial intelligence that focuses on the interaction between computers and humans through natural language. The ultimate objective of NLP is to read, decipher, understand, and make sense of human languages in a valuable way. |
| Word tokens (punctuation omitted) | Natural, Language, Processing, NLP, is, a, field, of, artificial, intelligence, that, focuses, on, the, interaction, between, computers, and, humans, through, natural, language, The, ultimate, objective, of, NLP, is, to, read, decipher, understand, and, make, sense, of, human, languages, in, a, valuable, way |
| Lowercase | natural, language, processing, nlp, is, a, field, of, artificial, intelligence, that, focuses, on, the, interaction, between, computers, and, humans, through, natural, language, the, ultimate, objective, of, nlp, is, to, read, decipher, understand, and, make, sense, of, human, languages, in, a, valuable, way |
| Declared stoplist | a, and, i, in, is, of, on, that, the, to |
| After exact stop removal | natural, language, processing, nlp, field, artificial, intelligence, focuses, interaction, between, computers, humans, through, natural, language, ultimate, objective, nlp, read, decipher, understand, make, sense, human, languages, valuable, way |

| Operation | Why it is useful |
| --- | --- |
| Sentence segmentation | Defines separate units for parsing and prevents counts from crossing sentence boundaries. |
| Lowercasing | Merges case variants when capitalization is not needed; retain original case for NER when useful. |
| Stop removal | Can remove frequent low-information words for a bag-of-words retrieval task; it is task-dependent and can damage negation/grammar. |
| Tokenization | Defines the units whose counts/features are computed; punctuation and abbreviation conventions must be consistent. |