Natural Language Processing · TEE · papers

Q4A · Bag of Words and three cosine similarities

Solved re-exam · February 2026

Actual re-examination · 9 February 2026 · 100 marks · 3 hours

Question. Consider the following three documents:
Document A: “The dog chased the cat.”
Document B: “The cat chased the mouse.”
Document C: “The dog ran after the cat.”
Using the Bag of Words (BoW) model, represent each document as a vector after removing the stop words {The, by, was}.
i. Construct the term-document matrix.
ii. Compute the cosine similarity between each pair of documents (A–B, A–C, and B–C).
iii. Interpret and comment on the results obtained in terms of document similarity. [10 marks]

Answer

Given

Three documents. Lowercase and remove punctuation. Remove the specified stop words case-insensitively; “the” is removed everywhere. Keep “after” because it is not in the supplied stop list. Use raw word counts.

To find

A common-vocabulary count matrix, three cosine values and their interpretation.

Formula: which one and why

\mathrm{BoW}_j(d)=C(v_j,d)LaTeX: \mathrm{BoW}_j(d)=C(v_j,d)

vⱼ is vocabulary term j. The coordinate stores its count in document d; order in the sentence is not retained.

\cos(\mathbf a,\mathbf b)=\frac{\sum_j a_jb_j}{\sqrt{\sum_j a_j^2}\sqrt{\sum_j b_j^2}}LaTeX: \cos(\mathbf a,\mathbf b)=\frac{\sum_j a_jb_j}{\sqrt{\sum_j a_j^2}\sqrt{\sum_j b_j^2}}

Solution

Step 1. Clean each document

A: dog chased cat.
B: cat chased mouse.
C: dog ran after cat.

Step 2. Fix vocabulary and matrix

Vocabulary: dog, chased, cat, mouse, ran, after.

TermABC
dog101
chased110
cat111
mouse010
ran001
after001

A = [1,1,1,0,0,0]; B = [0,1,1,1,0,0]; C = [1,0,1,0,1,1].

Step 3. Calculate vector lengths

\|A\|=\sqrt{1^2+1^2+1^2}=\sqrt3LaTeX: \|A\|=\sqrt{1^2+1^2+1^2}=\sqrt3
\|B\|=\sqrt{1^2+1^2+1^2}=\sqrt3LaTeX: \|B\|=\sqrt{1^2+1^2+1^2}=\sqrt3
\|C\|=\sqrt{1^2+1^2+1^2+1^2}=2LaTeX: \|C\|=\sqrt{1^2+1^2+1^2+1^2}=2

Step 4. Calculate A–B

A\cdot B=(1)(0)+(1)(1)+(1)(1)+(0)(1)+(0)(0)+(0)(0)=2LaTeX: A\cdot B=(1)(0)+(1)(1)+(1)(1)+(0)(1)+(0)(0)+(0)(0)=2
\cos(A,B)=\frac2{\sqrt3\sqrt3}=\frac23\approx0.666667LaTeX: \cos(A,B)=\frac2{\sqrt3\sqrt3}=\frac23\approx0.666667

Step 5. Calculate A–C

A\cdot C=(1)(1)+(1)(0)+(1)(1)+(0)(0)+(0)(1)+(0)(1)=2LaTeX: A\cdot C=(1)(1)+(1)(0)+(1)(1)+(0)(0)+(0)(1)+(0)(1)=2
\cos(A,C)=\frac2{\sqrt3\cdot2}=\frac1{\sqrt3}\approx0.577350LaTeX: \cos(A,C)=\frac2{\sqrt3\cdot2}=\frac1{\sqrt3}\approx0.577350

Step 6. Calculate B–C

B\cdot C=(0)(1)+(1)(0)+(1)(1)+(1)(0)+(0)(1)+(0)(1)=1LaTeX: B\cdot C=(0)(1)+(1)(0)+(1)(1)+(1)(0)+(0)(1)+(0)(1)=1
\cos(B,C)=\frac1{\sqrt3\cdot2}\approx0.288675LaTeX: \cos(B,C)=\frac1{\sqrt3\cdot2}\approx0.288675

Final answer

A–B = 0.666667; A–C = 0.577350; B–C = 0.288675. A and B have the highest lexical overlap under this representation. B and C share only cat. BoW similarity reflects token overlap, not full semantic equivalence or word order.

Yaad rakho: Same vocabulary order rakho. Corresponding counts multiply-add; phir dono lengths se divide.