# Drill 19 GD variation 2

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s19

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s19/

## Question

**Question.** J(θ) = ½(θ−1)², θ₀ = -0.5, η = 0.1. Perform two GD updates.

## Answer — step by step

### Given

Objective: J(θ) = ½(θ−1)².

Initial θ₀ = −0.5; η = 0.1; two iterations.

### To find

Find every stored state needed by the method and the parameter after each of two updates.

### Formula — which one and why

**Gradient descent.** Evaluate the gradient at the current parameter, multiply by the learning rate and subtract. If the objective is supplied, differentiate it and recompute the gradient after each update.

θ (theta) = trainable parameter; g = derivative of the stated objective J; η = learning rate; t labels the update number.



$$
g_t=J\prime(\theta_{t-1})
$$





$$
\theta_t=\theta_{t-1}-\eta g_t
$$



**Yaad rakhne ka cue:** Naya = purana − learning rate × current gradient.

### Solution — step by step

### Meaning of the symbols

θ (theta) is the number we update. η (eta) is learning rate 0.1. g is the derivative of the objective, evaluated at the required position. A subscript counts updates; θ₀ means the starting value.

### Differentiate the objective first



$$
J(\theta)=\frac12(\theta-1)^2,\qquad J\prime(\theta)=\theta-1
$$



The power rule brings down 2, which cancels the half. The inside derivative is 1, so the remaining gradient is θ minus 1.

### Update formulas



$$
g_t=J\prime(\theta_{t-1}),\quad \theta_t=\theta_{t-1}-\eta g_t
$$



**Step 1: Update 1.**



$$
g_1=-0.5-(1)=-1.5
$$





$$
\theta_1=-0.5-0.1(-1.5)=-0.35
$$



**Step 2: Update 2.**



$$
g_2=-0.35-(1)=-1.35
$$





$$
\theta_2=-0.35-0.1(-1.35)=-0.215
$$



**Final parameter after two updates: -0.215.** The second gradient is evaluated again using the parameter obtained after the first update.