Deep Learning · MT2 · formulas

Standardization

Standardization has a meaning before it has a formula: measure how far an input is from its training mean, then express that distance in standard-deviation units. The worked question below calculates the mean and standard deviation first, so neither number appears without explanation.

$x_{scaled}=\frac{x-\mu_{train}}{s_{train}}$ LaTeX: x_{scaled}=\frac{x-\mu_{train}}{s_{train}}

Standardization: what the answer 1 actually means

Definition. Standardization is an affine transformation that expresses an observation as its signed distance from a reference mean, measured in reference standard-deviation units.

Subtracting the mean centres the reference data. Dividing by its standard deviation rescales its spread. The result is dimensionless. A positive z lies above the training mean, a negative z below it, and z = 0 equals it. Standardization does not make an arbitrary distribution Gaussian and does not restrict values to [0,1].

Question. A feature has training values [3, 3, 7, 7]. Using population standard deviation, standardize the training values and a new observation x = 9.

Given

Training observations = [3,3,7,7].

New observation = 9.

Estimate the population standard deviation from the training values.

To find

Find the training mean and standard deviation; standardize all four training values and the new observation.

Formula — which one and why

Mean and spread. The mean is the sum divided by the number of values. Population variance averages squared distances from that mean; standard deviation is its square root.

N = number of values; xᵢ = value i; μ = mean; σ² = variance; σ = standard deviation. Σ means add the indicated terms.

$\mu=\frac{1}{N}\sum_{i=1}^{N}x_i$ LaTeX: \mu=\frac{1}{N}\sum_{i=1}^{N}x_i

$\sigma^2=\frac{1}{N}\sum_{i=1}^{N}(x_i-\mu)^2$ LaTeX: \sigma^2=\frac{1}{N}\sum_{i=1}^{N}(x_i-\mu)^2

$\sigma=\sqrt{\sigma^2}$ LaTeX: \sigma=\sqrt{\sigma^2}

Yaad rakhne ka cue: Mean = sabka total ÷ count; variance = squared distances ka mean.

Standardization. Subtract the training mean to measure distance from the centre; divide by the training standard deviation to express that distance in standard-deviation units.

x = original value; μ = training mean; σ = training standard deviation, which must be nonzero; z = standardized value.

$z=\frac{x-\mu}{\sigma}$ LaTeX: z=\frac{x-\mu}{\sigma}

Yaad rakhne ka cue: Pehle centre hatao, phir spread se divide karo.

Solution — step by step

Step 1: add all four observations.

$3+3+7+7=20$ LaTeX: 3+3+7+7=20

Step 2: divide by the count to obtain the mean.

$\mu=\frac{20}{4}=5$ LaTeX: \mu=\frac{20}{4}=5

Step 3: subtract the mean from each observation.

Step 4: square each deviation.

Step 5: add the squared deviations.

$4+4+4+4=16$ LaTeX: 4+4+4+4=16

Step 6: divide by four for population variance.

$\sigma^2=\frac{16}{4}=4$ LaTeX: \sigma^2=\frac{16}{4}=4

Step 7: take the nonnegative square root.

$\sigma=\sqrt4=2$ LaTeX: \sigma=\sqrt4=2

Step 8: standardize each original training observation.

$\frac{3-5}{2}=-1$ LaTeX: \frac{3-5}{2}=-1

$\frac{7-5}{2}=1$ LaTeX: \frac{7-5}{2}=1

The repeated observations give [-1, −1, 1, 1].

Step 9: subtract the training mean from the new observation.

$9-5=4$ LaTeX: 9-5=4

Step 10: divide by the training standard deviation.

$z_{\mathrm{new}}=\frac42=2$ LaTeX: z_{\mathrm{new}}=\frac42=2

Answer: the new observation is two training standard deviations above the training mean. Do not recompute the mean and standard deviation using the new observation; validation and test data use the training transformation to avoid leakage. A constant feature has zero standard deviation and needs a stated handling policy.