# Squared and absolute losses

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/formulas/13

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/formulas/13/

SSE, half SSE and MSE start with the same errors but apply different final divisions. First understand what an error and its square mean. The full question below keeps all three answers separate.



$$
\mathrm{SSE}=\sum_i(o_i-y_i)^2
$$





$$
L_{\mathrm{half}}=\frac{\mathrm{SSE}}2
$$





$$
L_{\mathrm{MSE}}=\frac{\mathrm{SSE}}n
$$





$$
L_{\mathrm{MAE}}=\frac1n\sum_i|o_i-y_i|
$$



## Loss: how one wrong prediction becomes a number

**Definition.** A residual is the signed prediction error. Squared-error objectives aggregate squared residuals; their scaling convention changes both the reported loss and its derivative.



$$
e_i=\hat y_i-y_i,\qquad \mathrm{SSE}=\sum_{i=1}^{n}e_i^2,\qquad L_{\mathrm{half}}=\frac12\sum_{i=1}^{n}e_i^2,\qquad \mathrm{MSE}=\frac1n\sum_{i=1}^{n}e_i^2
$$



For this definition of n, each i indexes one included scalar prediction. In a multi-output batch, state whether the mean is over examples, output coordinates, or both. Half SSE and MSE happen to agree when n = 2; they are not generally the same objective.

### Derivatives and their scaling

When differentiating with respect to prediction j, all other prediction terms are constant. The derivative of the j-th square is twice its residual:



$$
\frac{\partial\mathrm{SSE}}{\partial\hat y_j}=2e_j,\qquad \frac{\partial L_{\mathrm{half}}}{\partial\hat y_j}=e_j,\qquad \frac{\partial\mathrm{MSE}}{\partial\hat y_j}=\frac{2}{n}e_j
$$



The factor 1/2 cancels the derivative's factor 2. The factor 1/n instead produces 2/n. A weight derivative must also multiply by the derivative of the prediction with respect to that weight.

**Question. Predictions are [0.2, 0.7, 0.4] and targets are [0, 1, 1]. Find SSE, half SSE and MSE, then their gradients with respect to the prediction vector.**

### Given

Predictions = [0.2,0.7,0.4].

Targets = [0,1,1]; n = 3 scalar predictions.

### To find

Find SSE, half SSE and MSE, then all three prediction-gradient lists.

### Formula — which one and why

**Squared-error objectives.** Calculate signed errors first and square them. SSE adds the squares; half SSE divides that sum by 2; MSE divides by the number of scalar predictions. These denominators are different conventions.

ŷᵢ = prediction; yᵢ = target; eᵢ = signed error; n = number of scalar predictions included in the stated loss.



$$
e_i=\hat y_i-y_i
$$





$$
\mathrm{SSE}=\sum_i e_i^2
$$





$$
L_{\mathrm{half}}=\frac12\sum_i e_i^2
$$





$$
\mathrm{MSE}=\frac1n\sum_i e_i^2
$$



**Yaad rakhne ka cue:** Pehle error square; phir question ke hisaab se sum, half ya mean.

### Solution — step by step

**Step 1: calculate the three signed prediction errors.**



$$
e=[0.2-0,\ 0.7-1,\ 0.4-1]=[0.2,-0.3,-0.6]
$$



**Step 2: square every error.**



$$
e\odot e=[0.04,0.09,0.36]
$$



**Step 3: add the three squared errors.**



$$
\mathrm{SSE}=0.04+0.09+0.36=0.49
$$



**Step 4: apply the requested loss divisors.**



$$
L_{\mathrm{half}}=\frac{0.49}{2}=0.245,\qquad \mathrm{MSE}=\frac{0.49}{3}\approx0.163333333
$$



**Step 5: multiply every error by 2 for the SSE gradient.**



$$
\nabla_{\hat y}\mathrm{SSE}=[0.4,-0.6,-1.2]
$$



**Step 6: use each error itself for the half-SSE gradient.**



$$
\nabla_{\hat y}L_{\mathrm{half}}=[0.2,-0.3,-0.6]
$$



**Step 7: multiply every error by 2/3 for the MSE gradient.**



$$
\nabla_{\hat y}\mathrm{MSE}=\frac23[0.2,-0.3,-0.6]=[0.133333333,-0.2,-0.4]
$$



The residual signs survive in the gradients even though the losses are nonnegative. Multiplying a loss by a positive constant leaves its unregularized minimizers unchanged, but scales its gradients and therefore its updates at a fixed learning rate.

## Drill 2 Half sum versus mean

## Question

**Question.** Predictions = [0.2,0.7,0.4], targets = [0,1,1]. Find SSE, half SSE and MSE.

## Answer — step by step

### Given

Predictions = [0.2, 0.7, 0.4].

Targets = [0, 1, 1]; number of outputs n = 3.

### To find

Find SSE, half SSE, MSE and the prediction gradients requested in the question.

### Formula — which one and why

**Squared-error objectives.** Calculate signed errors first and square them. SSE adds the squares; half SSE divides that sum by 2; MSE divides by the number of scalar predictions. These denominators are different conventions.

ŷᵢ = prediction; yᵢ = target; eᵢ = signed error; n = number of scalar predictions included in the stated loss.



$$
e_i=\hat y_i-y_i
$$





$$
\mathrm{SSE}=\sum_i e_i^2
$$





$$
L_{\mathrm{half}}=\frac12\sum_i e_i^2
$$





$$
\mathrm{MSE}=\frac1n\sum_i e_i^2
$$



**Yaad rakhne ka cue:** Pehle error square; phir question ke hisaab se sum, half ya mean.

### Solution — step by step

### Given values and loss definitions

There are n = 3 prediction–target pairs. Use the supplied predictions directly; no rounding to class labels is required. Let eᵢ = ŷᵢ − yᵢ be the signed residual for pair i.



$$
\mathrm{SSE}=\sum_{i=1}^{n}e_i^2
$$





$$
L_{\mathrm{half}}=\frac12\sum_{i=1}^{n}e_i^2=\frac{\mathrm{SSE}}2
$$





$$
\mathrm{MSE}=\frac1n\sum_{i=1}^{n}e_i^2=\frac{\mathrm{SSE}}n
$$



SSE is the sum of squared errors. Half SSE and MSE rescale that same sum by different denominators: 2 and n, respectively.

### Step 1: calculate each residual and its square

| Pair | Prediction ŷᵢ | Target yᵢ | Residual ŷᵢ − yᵢ | Squared residual |
| --- | --- | --- | --- | --- |
| 1 | 0.2 | 0 | 0.2 − 0 = 0.2 | (0.2)² = 0.04 |
| 2 | 0.7 | 1 | 0.7 − 1 = −0.3 | (−0.3)² = 0.09 |
| 3 | 0.4 | 1 | 0.4 − 1 = −0.6 | (−0.6)² = 0.36 |

### Step 2: sum the squared residuals



$$
\mathrm{SSE}=0.04+0.09+0.36=0.49
$$



Square each residual before summing. Squaring the sum of signed residuals would be a different quantity, because positive and negative residuals could cancel.

### Step 3: apply each requested normalization

For half SSE, the denominator is 2:



$$
L_{\mathrm{half}}=\frac{0.49}{2}=0.245
$$



For MSE, the denominator is the number of pairs, n = 3:



$$
\mathrm{MSE}=\frac{0.49}{3}=0.163333333\ldots
$$



**Final answer:** SSE = 0.49; half SSE = 0.245; MSE ≈ 0.163333. They represent the same residuals at different scales. Half SSE and MSE coincide when n = 2; they do not coincide for these three nonzero residuals. No derivative is required by this question.

### Absolute error uses a different operation

**Question. Targets are [2, 4] and predictions are [1, 6]. Find mean absolute error.**

### Given

Targets = [2,4]; predictions = [1,6]; n = 2.

### To find

Find mean absolute error.

### Formula — which one and why

**Mean absolute error.** Absolute value removes the sign of each prediction error without squaring its magnitude. Average those magnitudes.

n = number of predictions. |e| means the nonnegative magnitude of error e.



$$
\mathrm{MAE}=\frac1n\sum_i|\hat y_i-y_i|
$$



**Yaad rakhne ka cue:** MAE mein magnitude; MSE mein square.

### Solution — step by step

The signed errors are 1 − 2 = −1 and 6 − 4 = 2. Absolute value keeps the size of a number without its sign, so their absolute errors are 1 and 2. Add them to obtain 3. There are two prediction–target pairs, so divide by 2.



$$
\mathrm{MAE}=\frac{|1-2|+|6-4|}{2}=\frac{1+2}{2}=1.5
$$



**Final answer: MAE = 1.5.** Do not square the errors for MAE.