Natural Language Processing · TEE · papers

Q2b · Lexical similarity and cosine

2024-12-03 · final · recovered AI paper

Previous examination · complete worked answers

Question. Explain the need for lexical similarity and illustrate cosine similarity. [4 marks]

Answer

Similarity supports retrieval, clustering, duplicate detection and semantic matching. Choose a representation first: character overlap measures spelling; context embeddings approximate semantic relatedness. An illustrative cosine computation follows; these vectors are chosen to explain the method, not given exam data.

Given

u=[1, 1, 0]; v=[1, 0, 1]

To find

Cosine similarity in the given coordinate order.

Formula: which one and why

\cos(u,v)=\frac{u\cdot v}{\|u\|\|v\|}LaTeX: \cos(u,v)=\frac{u\cdot v}{\|u\|\|v\|}

Step 1. Multiply matching coordinates

1×1 + 1×0 + 0×1 = 1

Step 2. Calculate both squared lengths

u: 1² + 1² + 0² = 2; v: 1² + 0² + 1² = 2

Step 3. Take square roots and divide

1 / (√2 × √2) = 0.500000000

Final answer

Cosine=0.500000000. This measures direction, not a probability.