# Q4a · NLP pipeline

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-r25/q4a

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-r25--q4a/

Previous examination · complete worked answers

**Question. Explain the stages of an NLP pipeline.** [5 marks]

### Answer

**An NLP pipeline is an organized sequence of operations that transforms raw language into a task output and checks whether that output is useful.** Not every task needs every cleaning operation.

Data acquisition

→

Preprocessing

→

Feature representation

→

Model

→

Evaluation

→

Deployment and monitoring

1. **Data acquisition:** collect relevant text and, when needed, labels.
2. **Cleaning and preprocessing:** handle markup, tokenize, normalize and decide which information to retain.
3. **Feature engineering:** represent text numerically using BoW, TF–IDF or embeddings.
4. **Modelling:** apply rules or train a statistical/neural model.
5. **Evaluation:** test on held-out data using task-appropriate measures.
6. **Deployment:** make the system available, log suitable feedback and monitor errors or data changes.

### Concrete spam-filter example

Collect labelled emails → separate train/test data → tokenize text while preserving meaningful terms → fit vocabulary and IDF on training data → train classifier → evaluate spam detection on the held-out set → deploy an email service filter → monitor legitimate emails wrongly blocked.

**Intrinsic evaluation** measures model/task quality, such as precision, recall or tagging accuracy. **Extrinsic evaluation** measures effect in the wider application, such as whether users find answers faster. A higher internal metric does not automatically guarantee a better user outcome.

**Data leakage:** learning vocabulary statistics, labels or preprocessing decisions from the test set can make reported performance too optimistic. Fit learned transformations on training data, then apply them to validation/test data.

Collect → clean → numbers → model → test → use. Test data ko training ki information mat banao.

**Definition.** NLP is the study of computational methods for analyzing, understanding and generating human language. The pipeline diagram above turns raw language into a task-specific prediction or generated response.