# The derivative of prediction error

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/3/tee-c3-s7

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c3-s7/

For the half squared loss, use prediction minus target. Let target be y and prediction be ŷ.

![The half and the averaging factor belong to the loss definition, so they also affect its derivative.](https://deep-learning-study-room.vercel.app/assets/figures/v_loss_gradient.svg)

Diagram: The half and the averaging factor belong to the loss definition, so they also affect its derivative.. Labels: Prediction error and its derivative | Prediction error and its derivative | Keep the target y fixed when differentiating. | Half squared loss | L = ½(ŷ − y)² | Derivative = ŷ − y | The 2 cancels the half. | Mean squared error | L = sum of squares ÷ n | Derivative per output | = (2/n)(ŷ − y) | The scale of the loss changes the scale of its gradient.

The half and the averaging factor belong to the loss definition, so they also affect its derivative. Tap the diagram to enlarge.



$$
L=\frac12(\hat y-y)^2,\qquad \frac{\partial L}{\partial \hat y}=\hat y-y
$$



The 2 from differentiating a square cancels the half. Writing the loss as ½(y − ŷ)² gives the same derivative: the derivative of the inside y − ŷ adds a minus sign. This is why “target minus prediction” without the extra minus gives the wrong gradient.

If the question uses plain squared error, its derivative is 2(ŷ − y). If it averages errors over n values, also divide by n. Read the stated loss before using a memorized gradient.