# Drill 6 Chain rule on a tiny loss

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s6

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s6/

## Question

**Question.** Let o = wx+b, L = ½(o−y)², x = 3, w = 0.2, b = 0.1, y = 1. Find dL/dw, dL/db and update at η = 0.1.

## Answer — step by step

### Given

Model: o = wx+b; loss L = ½(o−y)².

x = 3; y = 1; w = 0.2; b = 0.1.

Learning rate η = 0.1.

### To find

Find ∂L/∂w, ∂L/∂b and the updated weight and bias.

### Formula — which one and why

**Chain rule for a linear neuron.** The weight changes the prediction, and the prediction changes the loss. Multiply those two local rates. Half squared loss differentiates to prediction minus target; the neuron contributes input x.

z = prediction; y = target; e = z−y; g\_w and g\_b = loss derivatives for weight and bias. ∂L/∂w means “how fast loss changes when w changes”. η (eta) = learning rate.



$$
z=wx+b
$$





$$
L=\frac12(z-y)^2
$$





$$
g_w=\frac{\partial L}{\partial w}=(z-y)x
$$





$$
g_b=\frac{\partial L}{\partial b}=z-y
$$





$$
w_{\mathrm{new}}=w-\eta g_w
$$





$$
b_{\mathrm{new}}=b-\eta g_b
$$



**Yaad rakhne ka cue:** Ek path mein multiply; alag paths mil rahe hon toh add.

### Solution — step by step

### Meaning and formulas

The output o is a linear prediction: no sigmoid or ReLU is specified. Changing w changes o, which changes L. The chain rule multiplies those linked effects.



$$
o=wx+b,\qquad L=\frac12(o-y)^2
$$





$$
\frac{\partial L}{\partial o}=o-y,\quad \frac{\partial o}{\partial w}=x,\quad \frac{\partial o}{\partial b}=1
$$





$$
\frac{\partial L}{\partial w}=(o-y)x,\qquad \frac{\partial L}{\partial b}=o-y
$$



The learning rate η says how much of the gradient to subtract.



$$
w_{\mathrm{new}}=w-\eta\frac{\partial L}{\partial w},\qquad b_{\mathrm{new}}=b-\eta\frac{\partial L}{\partial b}
$$



### Step 1: obtain the current prediction and error



$$
o=(0.2\times3)+0.1=0.6+0.1=0.7
$$





$$
o-y=0.7-1=-0.3
$$





$$
L=\frac12(-0.3)^2=\frac12(0.09)=0.045
$$



### Step 2: calculate both gradients from this same prediction



$$
\frac{\partial L}{\partial w}=(-0.3)\times3=-0.9
$$





$$
\frac{\partial L}{\partial b}=(-0.3)\times1=-0.3
$$



### Step 3: update the weight



$$
w_{\mathrm{new}}=0.2-0.1(-0.9)=0.2-(-0.09)=0.29
$$



### Step 4: update the bias



$$
b_{\mathrm{new}}=0.1-0.1(-0.3)=0.1-(-0.03)=0.13
$$



**Final answer:** weight gradient −0.9; bias gradient −0.3; new weight 0.29; new bias 0.13. Both updates use the old forward pass.