# Standardization Subject: Deep Learning Exam: TEE Reader: https://deep-learning-study-room.vercel.app/#tee/formulas/2 Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/2/ Standardization has a meaning before it has a formula: measure how far an input is from its training mean, then express that distance in standard-deviation units. The worked question below calculates the mean and standard deviation first, so neither number appears without explanation. $$ x_{scaled}=\frac{x-\mu_{train}}{s_{train}} $$ ## Standardization: what the answer 1 actually means **Definition.** Standardization is an affine transformation that expresses an observation as its signed distance from a reference mean, measured in reference standard-deviation units. $$ z=\frac{x-\mu_{\mathrm{train}}}{\sigma_{\mathrm{train}}},\qquad \sigma_{\mathrm{train}}>0 $$ Subtracting the mean centres the reference data. Dividing by its standard deviation rescales its spread. The result is dimensionless. A positive z lies above the training mean, a negative z below it, and z = 0 equals it. Standardization does not make an arbitrary distribution Gaussian and does not restrict values to [0,1]. **Question. A feature has training values [3, 3, 7, 7]. Using population standard deviation, standardize the training values and a new observation x = 9.** ### Given Training observations = [3,3,7,7]. New observation = 9. Estimate the population standard deviation from the training values. ### To find Find the training mean and standard deviation; standardize all four training values and the new observation. ### Formula — which one and why **Mean and spread.** The mean is the sum divided by the number of values. Population variance averages squared distances from that mean; standard deviation is its square root. N = number of values; xᵢ = value i; μ = mean; σ² = variance; σ = standard deviation. Σ means add the indicated terms. $$ \mu=\frac{1}{N}\sum_{i=1}^{N}x_i $$ $$ \sigma^2=\frac{1}{N}\sum_{i=1}^{N}(x_i-\mu)^2 $$ $$ \sigma=\sqrt{\sigma^2} $$ **Yaad rakhne ka cue:** Mean = sabka total ÷ count; variance = squared distances ka mean. **Standardization.** Subtract the training mean to measure distance from the centre; divide by the training standard deviation to express that distance in standard-deviation units. x = original value; μ = training mean; σ = training standard deviation, which must be nonzero; z = standardized value. $$ z=\frac{x-\mu}{\sigma} $$ **Yaad rakhne ka cue:** Pehle centre hatao, phir spread se divide karo. ### Solution — step by step **Step 1: add all four observations.** $$ 3+3+7+7=20 $$ **Step 2: divide by the count to obtain the mean.** $$ \mu=\frac{20}{4}=5 $$ **Step 3: subtract the mean from each observation.** $$ [3-5,3-5,7-5,7-5]=[-2,-2,2,2] $$ **Step 4: square each deviation.** $$ [(-2)^2,(-2)^2,2^2,2^2]=[4,4,4,4] $$ **Step 5: add the squared deviations.** $$ 4+4+4+4=16 $$ **Step 6: divide by four for population variance.** $$ \sigma^2=\frac{16}{4}=4 $$ **Step 7: take the nonnegative square root.** $$ \sigma=\sqrt4=2 $$ **Step 8: standardize each original training observation.** $$ \frac{3-5}{2}=-1 $$ $$ \frac{7-5}{2}=1 $$ The repeated observations give [-1, −1, 1, 1]. **Step 9: subtract the training mean from the new observation.** $$ 9-5=4 $$ **Step 10: divide by the training standard deviation.** $$ z_{\mathrm{new}}=\frac42=2 $$ **Answer:** the new observation is two training standard deviations above the training mean. Do not recompute the mean and standard deviation using the new observation; validation and test data use the training transformation to avoid leakage. A constant feature has zero standard deviation and needs a stated handling policy. ### Standardization with x = 3 **Question. Training mean is 5 and training standard deviation is 2. Standardize x = 3 and interpret the result.** ### Given x = 3; training mean μ = 5; training standard deviation σ = 2. ### To find Standardized value z and its meaning. ### Formula — which one and why Subtract the training centre, then measure that distance in units of the training standard deviation. A negative z means below the training mean; positive means above it; zero means equal to it. $$ z=\frac{x-\mu}{\sigma} $$ **Yaad rakhne ka cue:** Centre hatao, spread se divide; answer batata hai kitne σ door. ### Solution — step by step **Step 1: find distance from the mean.** $$ x-\mu=3-5=-2 $$ **Step 2: divide by one standard-deviation unit.** $$ z=\frac{-2}{2}=-1 $$ **Step 3: interpret sign and size.** The value is 1 standard-deviation unit(s) below the training mean. **Final answer:** z = -1. Use the same training statistics when transforming a new test input. ### Standardization with x = 5 **Question. Training mean is 5 and training standard deviation is 2. Standardize x = 5 and interpret the result.** ### Given x = 5; training mean μ = 5; training standard deviation σ = 2. ### To find Standardized value z and its meaning. ### Formula — which one and why Subtract the training centre, then measure that distance in units of the training standard deviation. A negative z means below the training mean; positive means above it; zero means equal to it. $$ z=\frac{x-\mu}{\sigma} $$ **Yaad rakhne ka cue:** Centre hatao, spread se divide; answer batata hai kitne σ door. ### Solution — step by step **Step 1: find distance from the mean.** $$ x-\mu=5-5=0 $$ **Step 2: divide by one standard-deviation unit.** $$ z=\frac{0}{2}=0 $$ **Step 3: interpret sign and size.** The value is 0 standard-deviation unit(s) away from the training mean. **Final answer:** z = 0. Use the same training statistics when transforming a new test input. ### Standardization with x = 9 **Question. Training mean is 5 and training standard deviation is 2. Standardize x = 9 and interpret the result.** ### Given x = 9; training mean μ = 5; training standard deviation σ = 2. ### To find Standardized value z and its meaning. ### Formula — which one and why Subtract the training centre, then measure that distance in units of the training standard deviation. A negative z means below the training mean; positive means above it; zero means equal to it. $$ z=\frac{x-\mu}{\sigma} $$ **Yaad rakhne ka cue:** Centre hatao, spread se divide; answer batata hai kitne σ door. ### Solution — step by step **Step 1: find distance from the mean.** $$ x-\mu=9-5=4 $$ **Step 2: divide by one standard-deviation unit.** $$ z=\frac{4}{2}=2 $$ **Step 3: interpret sign and size.** The value is 2 standard-deviation unit(s) above the training mean. **Final answer:** z = 2. Use the same training statistics when transforming a new test input.