Deep Learning · TEE · section

Faculty Assignment2 Q4 · Batch linear regression update

Faculty Assignment 2 · worked practice

Question

Question. x=[1,2],y=[3,5],w=b=0,η=0.1;J=(1/(2m))Σ(wxᵢ+b−yᵢ)²,m=2. Compute both gradients and update. (Faculty assignment: no marks printed)

Answer — step by step

Given

Two examples, shared w,b, m=2.

To find

One simultaneous batch update.

Formula: which one and why

\frac{\partial J}{\partial w}=\frac1m\sum_ie_ix_i,\quad\frac{\partial J}{\partial b}=\frac1m\sum_ie_i,\quad e_i=wx_i+b-y_iLaTeX: \frac{\partial J}{\partial w}=\frac1m\sum_ie_ix_i,\quad\frac{\partial J}{\partial b}=\frac1m\sum_ie_i,\quad e_i=wx_i+b-y_i

Step 1. Predict and subtract targets

xywx+bee×x
130-3-3
250-5-10

Step 2. Average each gradient sum

dw=(−3−10)/2=−6.5;db=(−3−5)/2=−4.

Step 3. Update both from old parameters

w′=0−0.1×(−6.5)=0.65;b′=0−0.1×(−4)=0.4.

Final answer

w′=0.65,b′=0.4. New predictions1.05and1.7; do not updatew before computing theoldbgradient.