# The M2 formula sheet

Subject: Natural Language Processing

Exam: M2

Reader: https://deep-learning-study-room.vercel.app/nlp/#m2/formulas/formula-index/all

Canonical: https://deep-learning-study-room.vercel.app/read/nlp/m2/formulas/formula-index--all/

M2 · formulas with meaning · full examples follow

Read each formula with its meaning. The remaining lessons in this tab contain the complete worked examples for these methods. Use the same vocabulary, boundary, case and token-count conventions throughout one solution.

### N-gram count



$$
\max(T-n+1,0)
$$



Unpadded sequence of T tokens. Count only valid contiguous windows.

### Unigram MLE



$$
P(w)=C(w)/N
$$



N is the training-token total under the stated boundary convention.

### Bigram MLE



$$
P(w_i\mid w_{i-1})=C(w_{i-1},w_i)/C(w_{i-1})
$$



Denominator = history count.

### Trigram MLE



$$
P(w_i\mid w_{i-2},w_{i-1})=C(w_{i-2},w_{i-1},w_i)/C(w_{i-2},w_{i-1})
$$



Denominator = two-word history count.

### Add-one / Laplace



$$
P(w\mid h)=\frac{C(hw)+1}{C(h)+|V|}
$$



Add one to every possible vocabulary continuation.

### Sentence probability



$$
P(W)\approx\prod_iP(w_i\mid w_{i-1})
$$



Multiply in order; include an end factor if specified.

### Perplexity



$$
PP=P(W)^{-1/N}
$$



N is the number of predicted tokens/events used by the question.

### Emission



$$
P(w\mid t)=C(w,t)/C(t)
$$



Word given tag; count tokens with that tag.

### Transition



$$
P(t_j\mid t_i)=C(t_i\to t_j)/\sum_k C(t_i\to t_k)
$$



Next tag given previous tag; count outgoing transitions.

### HMM joint path



$$
P(W,T)=\pi_{t_1}b_{t_1}(w_1)\prod_{i=2}^{n}a_{t_{i-1},t_i}b_{t_i}(w_i)
$$



Multiply initial, emission and transition factors; add end probability if used.

### Tag sequences



$$
K^T
$$



K tags, T word positions; count before constraints.

### Viterbi



$$
\delta_i(t)=b_t(w_i)\max_s[\delta_{i-1}(s)a_{s,t}]
$$



Choose the largest predecessor score; store a backpointer.

### Lesk



$$
score(s)=|context\cap gloss(s)|
$$



Use the specified normalisation and overlap convention.

### Naive Bayes class prior



$$
P(c)=D_c/D
$$



D counts labelled examples; this is not the word-token total.

### Smoothed NB likelihood



$$
P(w\mid c)=\frac{C(w,c)+1}{N_c+|V|}
$$



N\_c counts training tokens in class c.

### NB decision



$$
\hat c=\arg\max_c P(c)\prod_iP(w_i\mid c)
$$



Select label with largest score; normalise only for posterior values.

**Yaad rakho:** Condition jis par hai, denominator usi ka; sequence mein multiply; options mein maximum.