# TEE formula sheet: vectors and applications

Subject: Natural Language Processing

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/nlp/#tee/formulas/tee-formula-index/all-tee

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/tee/formulas/tee-formula-index--all-tee/

TEE · used together with the M2 formula sheet

These formulas extend the M2 sheet to the complete TEE syllabus. Each coordinate, token-count convention and log base must be defined before calculation. Worked-example chapters follow this sheet.

### Type–token ratio



$$
TTR=\frac{\#\text{types}}{\#\text{tokens}}
$$



Distinct vocabulary types divided by all token occurrences. Often expressed as a percentage.

### Normalized term frequency



$$
TF(t,d)=\frac{C(t,d)}{\sum_w C(w,d)}
$$



The slides use count of the term divided by the document token total after the specified preprocessing.

### Inverse document frequency



$$
IDF(t)=\log_{10}\frac{D}{df(t)}
$$



D is the number of documents; df counts documents containing the term at least once. This is the course slide convention.

### TF–IDF



$$
TF\!\operatorname{-}\!IDF(t,d)=TF(t,d)IDF(t)
$$



Multiply the local frequency weight by the global rarity weight.

### Dot product



$$
a\cdot b=\sum_i a_ib_i
$$



Multiply corresponding coordinates, then add. Vectors need the same coordinate order.

### Vector length



$$
\|a\|_2=\sqrt{\sum_i a_i^2}
$$



Square each coordinate, add, then take the square root.

### Cosine similarity



$$
\cos(a,b)=\frac{\sum_i a_ib_i}{\sqrt{\sum_i a_i^2}\sqrt{\sum_i b_i^2}}
$$



Compare direction; undefined for a zero-length vector.

### Jaccard similarity



$$
J(A,B)=\frac{|A\cap B|}{|A\cup B|}
$$



Set membership, not repeated word counts. Jaccard distance = 1 − J.

### Manhattan distance



$$
d_1(a,b)=\sum_i|a_i-b_i|
$$



Add coordinate-wise absolute differences.

### Euclidean distance



$$
d_2(a,b)=\sqrt{\sum_i(a_i-b_i)^2}
$$



Square differences, add, square root.

### Minkowski distance



$$
d_p(a,b)=\left(\sum_i|a_i-b_i|^p\right)^{1/p}
$$



p≥1; Manhattan p=1, Euclidean p=2.

### Mean sentence embedding



$$
v_{sentence}=\frac1n\sum_{i=1}^n v_{w_i}
$$



Add word vectors coordinate by coordinate; divide each coordinate by n. This baseline loses word order.

### GloVe objective



$$
J=\sum_{i,j}f(X_{ij})(w_i^T\widetilde w_j+b_i+\widetilde b_j-\log X_{ij})^2
$$



X is a co-occurrence count. Compute the residual, square it and apply the weighting function. Use observed positive counts.

### Precision and recall



$$
P=\frac{TP}{TP+FP},\qquad R=\frac{TP}{TP+FN}
$$



Precision checks predicted positives; recall checks actual positives.

### F1 score



$$
F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN}
$$



Harmonic mean of precision and recall.

### Accuracy



$$
A=\frac{TP+TN}{TP+TN+FP+FN}
$$



All correct predictions divided by all predictions.

### Noisy-channel correction



$$
\hat w=\arg\max_w P(x\mid w)P(w)
$$



Rank intended word w using observed typo x; likelihood × prior.

### Unit-cost edit distance



$$
D(i,j)=\min\{D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+[a_i\ne b_j]\}
$$



Initialise first row/column by insertion/deletion counts; fill all cells using min.