Natural Language Processing · TEE · formulas

Cosine: products, lengths and division

Representation & similarity

Formulas explained through full worked examples

Cosine similarity compares the direction of two nonzero vectors. Multiply matching coordinates and add to obtain the dot product. To obtain each vector's norm, square its coordinates, add them and take a square root.

\cos(x,y)=\frac{\sum_i x_i y_i}{\sqrt{\sum_i x_i^2}\sqrt{\sum_i y_i^2}}.

The vectors must use the same coordinate/vocabulary order. For nonnegative BoW or TF–IDF vectors, the result lies from 0 to 1; for general real-valued vectors, it can range from −1 to 1. A zero vector makes this formula undefined.

Question. Find cosine similarity for x=[1,1] and y=[1,0].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,1] and y=[1,0].

Formula and why it applies

Use $\cos(x,y)=(x\cdot y)/(\|x\|\|y\|)$ because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 1 = 1; 1 × 0 = 0.

Step 2. Add products

1 + 0 = 1.

Step 3. Length of x

$\sqrt{(1)^2+(1)^2}=\sqrt{2}.$

Step 4. Length of y

$\sqrt{(1)^2+(0)^2}=\sqrt{1}.$

Step 5. Divide

$\frac{1}{\sqrt{2}\sqrt{1}}\approx 0.707107.$

Answer. Cosine≈0.707107.

Question. Find cosine similarity for x=[1,2] and y=[2,4].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,2] and y=[2,4].

Formula and why it applies

Use $\cos(x,y)=(x\cdot y)/(\|x\|\|y\|)$ because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 2 = 2; 2 × 4 = 8.

Step 2. Add products

2 + 8 = 10.

Step 3. Length of x

$\sqrt{(1)^2+(2)^2}=\sqrt{5}.$

Step 4. Length of y

$\sqrt{(2)^2+(4)^2}=\sqrt{20}.$

Step 5. Divide

$\frac{10}{\sqrt{5}\sqrt{20}}\approx 1.000000.$

Answer. Cosine≈1.000000.

Question. Find cosine similarity for x=[1,2,0] and y=[0,1,2].

Given

Nonzero vectors, matching coordinate order.

To find

Find cosine similarity for x=[1,2,0] and y=[0,1,2].

Formula and why it applies

Use $\cos(x,y)=(x\cdot y)/(\|x\|\|y\|)$ because direction similarity is required.

Step 1. Multiply matching coordinates

1 × 0 = 0; 2 × 1 = 2; 0 × 2 = 0.

Step 2. Add products

0 + 2 + 0 = 2.

Step 3. Length of x

$\sqrt{(1)^2+(2)^2+(0)^2}=\sqrt{5}.$

Step 4. Length of y

$\sqrt{(0)^2+(1)^2+(2)^2}=\sqrt{5}.$

Step 5. Divide

$\frac{2}{\sqrt{5}\sqrt{5}}\approx 0.400000.$

Answer. Cosine≈0.400000.

Proportional vectors [1,2] and [2,4] have cosine 1 but nonzero Euclidean distance. Direction similarity does not require identical magnitudes.

Cosine: matching products add upar; dono lengths multiply neeche.