# Unit 1 formulas: probability, Bayes, naive Bayes, Bayesian networks, GMM

Subject: Intelligent Systems II

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/is2/#tee/formulas/tee-formulas/unit1

Canonical: https://deep-learning-study-room.vercel.app/read/is2/tee/formulas/tee-formulas--unit1/

Probability, Bayes, GMM/EM, Markov chains and Bellman equations (Units 2–3 are in the M2 sheet)

## Conditions and notation for this revision sheet

Conditional probabilities require nonzero evidence probability. “Prior” means the class probability before the features; “likelihood” is the feature probability given a class; “posterior” is the class probability after evidence. Σ adds alternatives; Π multiplies factors; argmax returns the winning label. For Laplace smoothing, Nc is the class count and k is the number of possible values of the particular feature.

For the Jensen expression below, λ weights are nonnegative, sum to 1, and the logged arguments are positive. For a Gaussian, σ>0. GMM weights πk are nonnegative and sum to 1; γik or λki are responsibilities, not the mixing weights themselves.

Worked explanations: [Naive Bayes](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u1-nb--model/) · [CPTs and Bayesian networks](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u1-bn--semantics/) · [Mean, variance and likelihood](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u1-gmm--centre-spread/) · [EM update](https://deep-learning-study-room.vercel.app/read/is2/tee/read/u1-gmm--em/).

|  |  |
| --- | --- |
| Conditional | 

$$
P(a\mid b)=P(a\land b)/P(b)
$$

 |
| Product rule | 

$$
P(a\land b)=P(a\mid b)P(b)
$$

 |
| Marginalisation | 

$$
P(Y)=\sum_zP(Y,z)
$$

 |
| Normalisation | 

$$
\mathbf P(X\mid e)=\alpha\,\mathbf P(X,e),\;\alpha=1/\sum_xP(x,e)
$$

 |
| Bayes' rule | 

$$
P(A\mid B)=\dfrac{P(B\mid A)P(A)}{P(B\mid A)P(A)+P(B\mid\lnot A)P(\lnot A)}
$$

 |
| Naive Bayes | 

$$
v_{NB}=\arg\max_cP(c)\prod_iP(a_i\mid c)
$$

; Laplace: 

$$
\frac{n_{ic}+1}{N_c+k}
$$

 |
| BN chain rule | 

$$
P(x_1..x_n)=\prod_iP(x_i\mid parents(X_i))
$$

 |
| BN query | 

$$
P(X\mid e)=\alpha\sum_{\text{hidden}}\prod P(\cdot\mid parents)
$$

 |
| CPT size | Boolean node with k Boolean parents: independent probability parameters = 

$$
2^k
$$

 numbers |
| GMM density | 

$$
p(x)=\sum_k\pi_k\mathcal N(x\mid\mu_k,\sigma_k^2)
$$

 |
| Gaussian | 

$$
\mathcal N(x\mid\mu,\sigma^2)=\frac1{\sqrt{2\pi}\sigma}e^{-\frac{(x-\mu)^2}{2\sigma^2}}
$$

 |
| Log-likelihood | 

$$
\log L=\sum_i\log\sum_k\pi_k\mathcal N(x_i\mid\mu_k,\sigma_k^2)
$$

 |
| Jensen (log concave) | 

$$
\log\sum_k\lambda_ka_k\ge\sum_k\lambda_k\log a_k
$$

 |
| E-step | 

$$
\lambda_k^i=\pi_k\mathcal N(x_i\mid\mu_k,\sigma_k^2)/\sum_l\pi_l\mathcal N(x_i\mid\mu_l,\sigma_l^2)
$$

 |
| M-step | 

$$
\mu_k=\frac{\sum\lambda_k^ix_i}{\sum\lambda_k^i},\;\sigma_k^2=\frac{\sum\lambda_k^i(x_i-\mu_k)^2}{\sum\lambda_k^i},\;\pi_k=\frac{\sum_i\lambda_k^i}{n}
$$

 |
| GMM parameters | 1-D: 

$$
3K-1
$$

 (K = 5 → 14) |