Natural Language Processing · TEE · formulas

TEE formula sheet: vectors and applications

TEE · additional formulas

TEE · used together with the M2 formula sheet

These formulas extend the M2 sheet to the complete TEE syllabus. Each coordinate, token-count convention and log base must be defined before calculation. Worked-example chapters follow this sheet.

Type–token ratio

TTR=\frac{\#\text{types}}{\#\text{tokens}}

Distinct vocabulary types divided by all token occurrences. Often expressed as a percentage.

Normalized term frequency

TF(t,d)=\frac{C(t,d)}{\sum_w C(w,d)}

The slides use count of the term divided by the document token total after the specified preprocessing.

Inverse document frequency

IDF(t)=\log_{10}\frac{D}{df(t)}

D is the number of documents; df counts documents containing the term at least once. This is the course slide convention.

TF–IDF

TF\!\operatorname{-}\!IDF(t,d)=TF(t,d)IDF(t)

Multiply the local frequency weight by the global rarity weight.

Dot product

a\cdot b=\sum_i a_ib_i

Multiply corresponding coordinates, then add. Vectors need the same coordinate order.

Vector length

\|a\|_2=\sqrt{\sum_i a_i^2}

Square each coordinate, add, then take the square root.

Cosine similarity

\cos(a,b)=\frac{\sum_i a_ib_i}{\sqrt{\sum_i a_i^2}\sqrt{\sum_i b_i^2}}

Compare direction; undefined for a zero-length vector.

Jaccard similarity

J(A,B)=\frac{|A\cap B|}{|A\cup B|}

Set membership, not repeated word counts. Jaccard distance = 1 − J.

Manhattan distance

d_1(a,b)=\sum_i|a_i-b_i|

Add coordinate-wise absolute differences.

Euclidean distance

d_2(a,b)=\sqrt{\sum_i(a_i-b_i)^2}

Square differences, add, square root.

Minkowski distance

d_p(a,b)=\left(\sum_i|a_i-b_i|^p\right)^{1/p}

p≥1; Manhattan p=1, Euclidean p=2.

Mean sentence embedding

v_{sentence}=\frac1n\sum_{i=1}^n v_{w_i}

Add word vectors coordinate by coordinate; divide each coordinate by n. This baseline loses word order.

GloVe objective

J=\sum_{i,j}f(X_{ij})(w_i^T\widetilde w_j+b_i+\widetilde b_j-\log X_{ij})^2

X is a co-occurrence count. Compute the residual, square it and apply the weighting function. Use observed positive counts.

Precision and recall

P=\frac{TP}{TP+FP},\qquad R=\frac{TP}{TP+FN}

Precision checks predicted positives; recall checks actual positives.

F1 score

F_1=\frac{2PR}{P+R}=\frac{2TP}{2TP+FP+FN}

Harmonic mean of precision and recall.

Accuracy

A=\frac{TP+TN}{TP+TN+FP+FN}

All correct predictions divided by all predictions.

Noisy-channel correction

\hat w=\arg\max_w P(x\mid w)P(w)

Rank intended word w using observed typo x; likelihood × prior.

Unit-cost edit distance

D(i,j)=\min\{D(i-1,j)+1,D(i,j-1)+1,D(i-1,j-1)+[a_i\ne b_j]\}

Initialise first row/column by insertion/deletion counts; fill all cells using min.