# Q3(a) GMM versus K-means [10]

Subject: Intelligent Systems II

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/is2/#tee/papers/paper-2024-final/q3a

Canonical: https://deep-learning-study-room.vercel.app/read/is2/tee/papers/paper-2024-final--q3a/

Actual historical paper ·100 marks attempted /140 offered · Q1 compulsory + any four of Q2–Q7. Source defects and solution assumptions are explicitly identified.

**Question. Q3(a)** [10 marks · 2024-04-17]

Discuss the significance of using Gaussian Mixture Models (GMMs) over K-means clustering in capturing complex data distributions.

### Answer

**GMM** models the density as a weighted sum of Gaussian components. **K-means** minimizes within-cluster squared Euclidean distance and assigns each point to one nearest centroid.



$$
p(x)=\sum_{k=1}^K\pi_k\mathcal N(x\mid\mu_k,\Sigma_k)
$$



| Aspect | K-means | Gaussian mixture |
| --- | --- | --- |
| Assignment | Hard: one cluster label | Soft: posterior responsibility for each component |
| Cluster shape | Voronoi regions from Euclidean distance; suited to compact roughly spherical groups | Full covariance allows elongated and differently oriented ellipses |
| Spread | No explicit cluster-specific covariance in the standard objective | Separate covariance estimates model unequal spreads |
| Overlap | Boundary points assigned completely to one side | Uncertainty is represented by partial memberships |
| Output | Centroids and assignments | Density, mixing weights, means, covariances and responsibilities |
| Training | Alternate assignment and centroid mean | EM responsibility and parameter updates |
| Cost | Usually simpler and faster | More parameters and cost; can overfit or develop singular covariance |
| Initialization | Can reach local minima | Can reach local likelihood maxima; multiple restarts help |

**Example.** Two customer groups overlap in annual income/spending: one is a narrow diagonal band and one a broad ellipse. A spherical distance-only partition may split the diagonal group badly. Separate covariance matrices can align GMM ellipses with each group. A boundary customer might have responsibilities [0.55,0.45], showing uncertainty instead of an unjustifiably certain label.

**Why this matters.** The mixture can approximate multimodal distributions; unequal mixing weights represent different group sizes. Its density supports likelihood-based anomaly detection and probabilistic downstream decisions. However a Gaussian component is not automatically a real-world class, and arbitrary non-Gaussian shapes may require several components.

**Selection and limitations.** Choose K and covariance restrictions using validation/BIC/AIC as appropriate, scale features, regularize covariance and inspect convergence. K-means remains attractive for large, roughly spherical, similarly scaled clusters when hard assignments are enough. GMM is more flexible, not universally superior.