# Q6a · GloVe

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/papers/legacy-f22/q6a

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/papers/legacy-f22--q6a/

Previous examination · complete worked answers

**Question. Explain GloVe word embeddings.** [10 marks]

### Answer

**GloVe (Global Vectors) learns word embeddings by fitting weighted relationships in a global word–context co-occurrence matrix.** Word2Vec uses local prediction examples; GloVe uses aggregate counts directly in its objective.



$$
X_i=\sum_k X_{ik},\qquad P_{ik}=\frac{X_{ik}}{X_i}.
$$





$$
X_{ik}
$$

 is how often context word 

$$
k
$$

 appears around target 

$$
i
$$

. 

$$
X_i
$$

 is the total of row 

$$
i
$$

. 

$$
P_{ik}
$$

 is the resulting context probability.

The ratio 

$$
P(k\mid i)/P(k\mid j)
$$

 helps identify how a probe context differs between two words. For targets ice and steam, a probe such as solid should favour ice; gas should favour steam; a probe shared similarly by both gives a ratio near one. These are data tendencies, not universal numerical identities.

**Question. For context solid, ice has count 12 out of row total 20; steam has count 2 out of row total 40. Compute P(solid|ice), P(solid|steam) and their ratio.**

#### Given

X(ice,solid)=12, X(ice)=20; X(steam,solid)=2, X(steam)=40.

#### To find

Two conditional probabilities and the ice/steam ratio.

#### Formula and why it applies



$$
P(k\mid i)=X_{ik}/X_i
$$

, and 

$$
R=P(k\mid ice)/P(k\mid steam)
$$

.

#### Step 1. Ice probability



$$
12/20=0.6
$$

.

#### Step 2. Steam probability



$$
2/40=0.05
$$

.

#### Step 3. Ratio



$$
0.6/0.05=12
$$

.

**Answer.** The probe solid is 12 times as probable in ice's context under these counts.

**Question. For probe water, ice and steam both have row total 50. Counts are 10 and 10. Calculate the ratio.**

#### Given

Both probabilities use a denominator of 50.

#### To find

The probability ratio.

#### Formula and why it applies

Use row-normalized counts before taking the ratio.

#### Step 1. Ice probability



$$
10/50=0.2
$$

.

#### Step 2. Steam probability



$$
10/50=0.2
$$

.

#### Step 3. Divide



$$
0.2/0.2=1
$$

.

**Answer.** Ratio 1: this probe does not distinguish the targets in this toy data.

**Question. For probe gas, counts are 1/100 around ice and 20/100 around steam. Compute the ice-to-steam ratio.**

#### Given

Conditional counts have the same row denominator.

#### To find

The ratio and interpretation.

#### Formula and why it applies



$$
R=P(gas\mid ice)/P(gas\mid steam)
$$

.

#### Step 1. Ice probability



$$
1/100=0.01
$$

.

#### Step 2. Steam probability



$$
20/100=0.2
$$

.

#### Step 3. Ratio



$$
0.01/0.2=0.05
$$

.

**Answer.** Ratio 0.05, so gas is much more strongly associated with steam.

Ratio >1: pehla word favoured. <1: doosra. ≈1: probe dono mein similar.