Natural Language Processing · TEE · papers

Q3b · Classification and sentiment in industry

2024-12-03 · final · recovered AI paper

Previous examination · complete worked answers

Question. Discuss real-world text classification and sentiment analysis, with examples showing decision-making and customer engagement. [10 marks]

Answer

Text classification assigns a document to one or more predefined categories. Spam detection usually uses two classes; news classification may use several topics. In single-label multiclass classification, choose one of several categories. In multilabel classification, several labels may apply to the same document.

Define labels

Prepare labelled examples

Train representation + model

Evaluate on unseen text

Predict new labels

An exam-ready pipeline

Stage	What you do	Why it matters
Task definition	Decide what counts as spam or which news topics exist.	Unclear labels create inconsistent training data.
Data collection and annotation	Collect representative examples and assign correct labels.	The model learns from these examples.
Split data	Separate training, validation and test data.	Test examples must not influence training choices.
Preprocess	Tokenize and normalise where suitable; retain meaningful negation.	Reduce irrelevant variation without deleting signal.
Represent	Use BoW, TF-IDF or learned embeddings.	Models need numerical features.
Train	Fit Naive Bayes, logistic regression or another appropriate model.	Learn how features predict categories.
Evaluate	Measure relevant precision, recall and F1 on held-out examples.	Accuracy alone may hide poor minority-class performance.
Deploy and monitor	Apply the same preprocessing and monitor changing language.	Incoming data can differ from old examples.

Naive Bayes is especially useful for hand calculations. With a bag-of-words representation, its class score is the prior multiplied by the conditional probability of each token; repeated tokens repeat the factor. The “naive” assumption is conditional independence of features given the class, not that words are independent in real language.

s(c)=P(c)\prod_{w\in V}P(w\mid c)^{C(w,d)}

P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|}

C(w,d) counts word w in document d. C(w,c) counts it in training documents belonging to class c. N_c is the total token count for class c. Add-one smoothing adds one to every vocabulary count and therefore adds |V| to the denominator. This is the same arithmetic taught for WSD, with document classes instead of senses.

Worked classification · two-token message

A spam classifier has vocabulary {win, prize, meeting}. Priors are P(spam)=P(ham)=0.5. Spam token counts are [3,2,0]; ham counts are [0,0,5]. With add-one smoothing, classify “win prize”.

Given

|V|=3; N(spam)=5; N(ham)=5; message has win once and prize once.

To find

Class scores and predicted label.

Formula and why it applies

s(c)=P(c)P(\text{win}\mid c)P(\text{prize}\mid c)

P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|}

The model is multinomial Naive Bayes; multiply the two token likelihoods and the class prior.

Solution

Step 1. Compute the smoothed denominators

5+3=8

Step 2. Compute spam likelihoods

P(\text{win}\mid S)=\frac{3+1}{8}=\frac12

P(\text{prize}\mid S)=\frac{2+1}{8}=\frac38

Step 3. Compute ham likelihoods

P(\text{win}\mid H)=P(\text{prize}\mid H)=\frac{0+1}{8}=\frac18

Step 4. Multiply the spam score

s(S)=\frac12\times\frac12\times\frac38=\frac{3}{32}=0.09375

Step 5. Multiply the ham score

s(H)=\frac12\times\frac18\times\frac18=\frac{1}{128}=0.0078125

Step 6. Compare

0.09375 > 0.0078125. These are unnormalised scores; the label is chosen by the larger score.

Predicted label: spam.

Yaad rakho: Class prior × har word ki class-likelihood; sabse bada score wins.

Sentiment analysis identifies expressed opinion or attitude, often as positive, negative or neutral. Document-level analysis labels a complete text; sentence-level analysis labels a sentence; aspect-level analysis attaches sentiment to a specific feature or entity.

Review	Useful interpretation
“The display is excellent, but the battery is poor.”	Display: positive; battery: negative. One overall label loses this distinction.
“The service is not good.”	Negative: negation changes good. Removing not can reverse the intended meaning.
“Great, another two-hour delay.”	Likely sarcasm: literal positive word alone is insufficient.

Approaches

Lexicon based: look up sentiment scores, combine them, and apply stated handling for negation/intensifiers. It is interpretable but fragile with context and sarcasm. Supervised: learn a mapping from labelled text features to sentiment labels. Neural/contextual: learn or use contextual representations to capture word interactions, then fine-tune a sentiment classifier. None guarantees correct interpretation of every review.

A specified lexicon calculation

Use good = +2, excellent = +3 and slow = −2. A “not” immediately before a sentiment word reverses its score. Score “The support was not good but the delivery was excellent and slow.”

Given

Three sentiment words; not modifies good only; sum word scores under the stated rule.

To find

Total sentiment score and label, taking positive totals as positive.

Formula and why it applies

S=\sum_i s_i

Apply the given negation rule before summing. This is a simplified specified model, not a universal linguistic rule.

Solution

Step 1. Identify sentiment words

good, excellent and slow.

Step 2. Apply negation

s(\text{not good})=-2

Step 3. Add the contributions

S=-2+3-2=-1

Step 4. Map score to label

The sum is negative.

Total score = −1; negative under this rule.

Industry	Concrete input and output	Useful decision / limitation
Retail	Review→topic and sentiment	Route delivery complaints; sarcasm/negation may mislead
Banking	Customer message→card/fraud/loan intent	Route to the correct team; a model flag is not proof of fraud
Healthcare	Feedback→service-quality topics	Detect recurring appointment issues; protect patient identifiers
Media	Article→topic; public responses→sentiment	Organize feeds and monitor reaction; sampling bias prevents treating posts as a census