Natural Language Processing · TEE · papers

Q4b · Text classification pipeline

2022-11-22 · final · recovered AI paper

Previous examination · complete worked answers

Question. Explain the text-classification pipeline with a diagram. [10 marks]

Answer

Text classification assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document.

Define labels

Prepare labelled examples

Train representation + model

Evaluate on unseen text

Predict new labels

An exam-ready pipeline

Stage	What you do	Why it matters
Task definition	Decide what counts as spam or which news topics exist.	Unclear labels create inconsistent training data.
Data collection and annotation	Collect representative examples and assign correct labels.	The model learns from these examples.
Split data	Separate training, validation and test data.	Test examples must not influence training choices.
Preprocess	Tokenize and normalise where suitable; retain meaningful negation.	Reduce irrelevant variation without deleting signal.
Represent	Use BoW, TF-IDF or learned embeddings.	Models need numerical features.
Train	Fit Naive Bayes, logistic regression or another appropriate model.	Learn how features predict categories.
Evaluate	Measure relevant precision, recall and F1 on held-out examples.	Accuracy alone may hide poor minority-class performance.
Deploy and monitor	Apply the same preprocessing and monitor changing language.	Incoming data can differ from old examples.

Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language.

s(c)=P(c)\prod_{w\in V}P(w\mid c)^{C(w,d)}

P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|}

C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. N_c is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses.

Worked classification · two-token message

A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”.

Given

|V|=3; N(spam)=5; N(ham)=5; message has win once and prize once.

To find

Class scores and predicted label.