Deep Learning · TEE · section

R24 Q2a

Re examination February 2024 fully solved

Question

Question (10 marks). A 4 → 4 → 1 sigmoid network has input [.4,.3,.5,−.4]. The printed W₁ is shown below, W₂ = [.5,.7,−.6,.2]; hidden bias 0.6, output bias 0.2, target 0.9, η = .5. Compute mean squared error and update the connection h₁ → output. In the graph θ ij connects input i to hidden j.

Given matrices for this question Tap the diagram to enlarge.
R24 Q2a receiving-neuron rows after transposing the source convention Tap the diagram to enlarge.

Answer — step by step

Given

A 4 → 4 → 1 sigmoid network has input [.4,.3,.5,−.4].

The printed W₁ is printed with the question, W₂ = [.5,.7,−.6,.2].

hidden bias 0.6, output bias 0.2, target 0.9, η = .5.

To find

Find all preactivations, hidden activations and final outputs. Calculate MSE and its output derivatives. Find the sigmoid output and, if requested, its derivative. Find the requested deltas and parameter gradients, then update the specified parameters.

Formula — which one and why

Forward propagation. Evaluate one layer completely before supplying its activated outputs to the next layer. Each layer has its own weights and biases.

x = input vector; h = hidden outputs; W₁,b₁ = hidden parameters; W₂,b₂ = output parameters; f₁,f₂ = the stated activations.

z_h=W_1x+b_1LaTeX: z_h=W_1x+b_1
h=f_1(z_h)LaTeX: h=f_1(z_h)
z_o=W_2h+b_2LaTeX: z_o=W_2h+b_2
o=f_2(z_o)LaTeX: o=f_2(z_o)

Yaad rakhne ka cue: Pehli layer ka activated output hi agli layer ka input hai.

Mean squared loss. Average the squared scalar prediction errors. The derivative retains the factor 2/n; n must count the outputs included in this mean.

n = number of scalar prediction errors in the stated mean; oⱼ = prediction; yⱼ = target.

L=\frac1n\sum_j(o_j-y_j)^2LaTeX: L=\frac1n\sum_j(o_j-y_j)^2
\frac{\partial L}{\partial o_j}=\frac2n(o_j-y_j)LaTeX: \frac{\partial L}{\partial o_j}=\frac2n(o_j-y_j)

Yaad rakhne ka cue: MSE mein derivative 2/n times error; half SSE ka rule blindly mat lagao.

Sigmoid and its slope. Use sigmoid when the neuron is specified to squash its preactivation into (0,1). Once its output is known, its derivative needs only output × (1−output).

z = preactivation; p = sigmoid output; e is the exponential base, not a prediction error in this formula.

p=\frac{1}{1+e^{-z}}LaTeX: p=\frac{1}{1+e^{-z}}
\frac{dp}{dz}=p(1-p)LaTeX: \frac{dp}{dz}=p(1-p)

Yaad rakhne ka cue: Sigmoid output p; uska slope p × (1−p).

Backpropagation. The loss tells us how each output must change. Multiply by that neuron’s activation slope to obtain its delta. A weight gradient is receiving delta × sending value. Hidden deltas must include every outgoing path.

δ (delta) = derivative of loss with respect to a neuron’s preactivation; f′ = activation derivative; aᵢ = sending activation; Wⱼᵢ = connection weight. Use the exact loss scaling in the question.

\delta_{o_j}=\frac{\partial L}{\partial o_j}f_o\prime(z_{o_j})LaTeX: \delta_{o_j}=\frac{\partial L}{\partial o_j}f_o\prime(z_{o_j})
\delta_{h_i}=f_h\prime(z_{h_i})\sum_jW_{ji}^{(2)}\delta_{o_j}LaTeX: \delta_{h_i}=f_h\prime(z_{h_i})\sum_jW_{ji}^{(2)}\delta_{o_j}
\frac{\partial L}{\partial W_{ji}}=\delta_ja_iLaTeX: \frac{\partial L}{\partial W_{ji}}=\delta_ja_i
\frac{\partial L}{\partial b_j}=\delta_jLaTeX: \frac{\partial L}{\partial b_j}=\delta_j
W_{ji,\mathrm{new}}=W_{ji}-\eta\frac{\partial L}{\partial W_{ji}}LaTeX: W_{ji,\mathrm{new}}=W_{ji}-\eta\frac{\partial L}{\partial W_{ji}}

Yaad rakhne ka cue: Receiving delta × sending value = us connection ka gradient.

Solution — step by step

Orientation: the source rows list sending inputs, so use each COLUMN to form a receiving hidden neuron. In our usual column-input convention this means using the transpose of printed W₁. The loss is one-output MSE = (o−0.9)², so its derivative includes 2.

Step 1 map the arrows. Each row of W₁ receives all inputs; each row of W₂ receives all hidden outputs. Bias is added once per receiving neuron. All gradients below use the OLD weights from this same forward pass.

Hidden neuron 1:

z_{h_1} = (0.8 \times 0.4) + (-0.5 \times 0.3) + (0.1 \times 0.5) + (-0.2 \times -0.4) + (0.6) = 0.9LaTeX: z_{h_1} = (0.8 \times 0.4) + (-0.5 \times 0.3) + (0.1 \times 0.5) + (-0.2 \times -0.4) + (0.6) = 0.9
h_{1}=\frac{1}{1+e^{-(0.9)}}=0.710949503LaTeX: h_{1}=\frac{1}{1+e^{-(0.9)}}=0.710949503

Hidden neuron 2:

z_{h_2} = (0.2 \times 0.4) + (0.4 \times 0.3) + (0.2 \times 0.5) + (0.5 \times -0.4) + (0.6) = 0.7LaTeX: z_{h_2} = (0.2 \times 0.4) + (0.4 \times 0.3) + (0.2 \times 0.5) + (0.5 \times -0.4) + (0.6) = 0.7
h_{2}=\frac{1}{1+e^{-(0.7)}}=0.668187772LaTeX: h_{2}=\frac{1}{1+e^{-(0.7)}}=0.668187772

Hidden neuron 3:

z_{h_3} = (0.3 \times 0.4) + (0.3 \times 0.3) + (-0.1 \times 0.5) + (0.5 \times -0.4) + (0.6) = 0.56LaTeX: z_{h_3} = (0.3 \times 0.4) + (0.3 \times 0.3) + (-0.1 \times 0.5) + (0.5 \times -0.4) + (0.6) = 0.56
h_{3}=\frac{1}{1+e^{-(0.56)}}=0.63645254LaTeX: h_{3}=\frac{1}{1+e^{-(0.56)}}=0.63645254

Hidden neuron 4:

z_{h_4} = (0.4 \times 0.4) + (0.2 \times 0.3) + (0.2 \times 0.5) + (0.8 \times -0.4) + (0.6) = 0.6LaTeX: z_{h_4} = (0.4 \times 0.4) + (0.2 \times 0.3) + (0.2 \times 0.5) + (0.8 \times -0.4) + (0.6) = 0.6
h_{4}=\frac{1}{1+e^{-(0.6)}}=0.645656306LaTeX: h_{4}=\frac{1}{1+e^{-(0.6)}}=0.645656306

Output neuron 1:

\begin{aligned}z_{o_1}={}&0.5(0.710949503)+0.7(0.668187772)\\&-0.6(0.63645254)+0.2(0.645656306)+0.2\\={}&0.770465929\end{aligned}LaTeX: \begin{aligned}z_{o_1}={}&0.5(0.710949503)+0.7(0.668187772)\\&-0.6(0.63645254)+0.2(0.645656306)+0.2\\={}&0.770465929\end{aligned}
o_{1}=\frac{1}{1+e^{-(0.770465929)}}=0.683621675LaTeX: o_{1}=\frac{1}{1+e^{-(0.770465929)}}=0.683621675

Step 2 loss convention. Use the explicitly requested one-output MSE, not half squared loss.

Prediction minus target = [-0.216378325]. Squared errors = [0.04681958]. Half-sum squared loss = 0.02340979; mean squared error = 0.04681958. These are different unless the number of output components makes the scale coincide.

Output delta 1:

\delta_{o_1} = 2 \times (0.683621675-0.9) \times 0.216283081 = -0.093597941LaTeX: \delta_{o_1} = 2 \times (0.683621675-0.9) \times 0.216283081 = -0.093597941

The derivative is o(1−o).

Step 3 output weights. Gradient = receiving delta × sending activation. An output bias gradient equals its delta.

h1 → o1:

The calculation on h1 → o1
Sending activation0.710949503h1
Receiving delta−0.093597941δ at o1
Connection gradient−0.06654341h1 → o1
Now update this weight
Old weight0.5
New weight0.533271705

Subtract η × gradient = −0.033271705. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies its activation.

g_{1,1}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,1}}=\delta_{o_1}h_1LaTeX: g_{1,1}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,1}}=\delta_{o_1}h_1
g_{1,1}^{(2)}=(-0.093597941)\times(0.710949503)\approx -0.06654341LaTeX: g_{1,1}^{(2)}=(-0.093597941)\times(0.710949503)\approx -0.06654341

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(2)}_{1,1,\mathrm{new}}=0.5-0.5(-0.06654341)\approx 0.533271705LaTeX: W^{(2)}_{1,1,\mathrm{new}}=0.5-0.5(-0.06654341)\approx 0.533271705

h2 → o1:

The calculation on h2 → o1
Sending activation0.668187772h2
Receiving delta−0.093597941δ at o1
Connection gradient−0.062541h2 → o1
Now update this weight
Old weight0.7
New weight0.7312705

Subtract η × gradient = −0.0312705. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies its activation.

g_{1,2}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,2}}=\delta_{o_1}h_2LaTeX: g_{1,2}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,2}}=\delta_{o_1}h_2
g_{1,2}^{(2)}=(-0.093597941)\times(0.668187772)\approx -0.062541LaTeX: g_{1,2}^{(2)}=(-0.093597941)\times(0.668187772)\approx -0.062541

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(2)}_{1,2,\mathrm{new}}=0.7-0.5(-0.062541)\approx 0.7312705LaTeX: W^{(2)}_{1,2,\mathrm{new}}=0.7-0.5(-0.062541)\approx 0.7312705

h3 → o1:

The calculation on h3 → o1
Sending activation0.63645254h3
Receiving delta−0.093597941δ at o1
Connection gradient−0.059570648h3 → o1
Now update this weight
Old weight−0.6
New weight−0.570214676

Subtract η × gradient = −0.029785324. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies its activation.

g_{1,3}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,3}}=\delta_{o_1}h_3LaTeX: g_{1,3}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,3}}=\delta_{o_1}h_3
g_{1,3}^{(2)}=(-0.093597941)\times(0.63645254)\approx -0.059570648LaTeX: g_{1,3}^{(2)}=(-0.093597941)\times(0.63645254)\approx -0.059570648

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(2)}_{1,3,\mathrm{new}}=-0.6-0.5(-0.059570648)\approx -0.570214676LaTeX: W^{(2)}_{1,3,\mathrm{new}}=-0.6-0.5(-0.059570648)\approx -0.570214676

h4 → o1:

The calculation on h4 → o1
Sending activation0.645656306h4
Receiving delta−0.093597941δ at o1
Connection gradient−0.060432101h4 → o1
Now update this weight
Old weight0.2
New weight0.230216051

Subtract η × gradient = −0.03021605. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies its activation.

g_{1,4}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,4}}=\delta_{o_1}h_4LaTeX: g_{1,4}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,4}}=\delta_{o_1}h_4
g_{1,4}^{(2)}=(-0.093597941)\times(0.645656306)\approx -0.060432101LaTeX: g_{1,4}^{(2)}=(-0.093597941)\times(0.645656306)\approx -0.060432101

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(2)}_{1,4,\mathrm{new}}=0.2-0.5(-0.060432101)\approx 0.230216051LaTeX: W^{(2)}_{1,4,\mathrm{new}}=0.2-0.5(-0.060432101)\approx 0.230216051

Step 4 hidden deltas. A hidden unit can influence EVERY connected output, so add those paths BEFORE multiplying by the hidden activation derivative.

Hidden 1 downstream contribution:

Calculate each outgoing path using the old outgoing weight and that output's delta.

Several output paths meet at this hidden neuron
Output 1
Output delta−0.093597941
Old outgoing weight0.5
Path contribution−0.04679897
Add every path contribution−0.046798971Then multiply by the hidden activation derivative.

Multiply within each path. Add the resulting contributions at the shared hidden neuron.

Contribution from output 1.

C_1=(0.5)\times(-0.093597941)\approx -0.04679897LaTeX: C_1=(0.5)\times(-0.093597941)\approx -0.04679897

Accumulate the contributions, then apply the hidden activation derivative below.

\text{Sum}=C_1\approx -0.046798971LaTeX: \text{Sum}=C_1\approx -0.046798971
\delta_{h_1} = -0.046798971 \times 0.205500307 = -0.009617203LaTeX: \delta_{h_1} = -0.046798971 \times 0.205500307 = -0.009617203

Hidden 2 downstream contribution:

Calculate each outgoing path using the old outgoing weight and that output's delta.

Several output paths meet at this hidden neuron
Output 1
Output delta−0.093597941
Old outgoing weight0.7
Path contribution−0.065518559
Add every path contribution−0.065518559Then multiply by the hidden activation derivative.

Multiply within each path. Add the resulting contributions at the shared hidden neuron.

Contribution from output 1.

C_1=(0.7)\times(-0.093597941)\approx -0.065518559LaTeX: C_1=(0.7)\times(-0.093597941)\approx -0.065518559

Accumulate the contributions, then apply the hidden activation derivative below.

\text{Sum}=C_1\approx -0.065518559LaTeX: \text{Sum}=C_1\approx -0.065518559
\delta_{h_2} = -0.065518559 \times 0.221712873 = -0.014526308LaTeX: \delta_{h_2} = -0.065518559 \times 0.221712873 = -0.014526308

Hidden 3 downstream contribution:

Calculate each outgoing path using the old outgoing weight and that output's delta.

Several output paths meet at this hidden neuron
Output 1
Output delta−0.093597941
Old outgoing weight−0.6
Path contribution0.056158765
Add every path contribution0.056158765Then multiply by the hidden activation derivative.

Multiply within each path. Add the resulting contributions at the shared hidden neuron.

Contribution from output 1.

C_1=(-0.6)\times(-0.093597941)\approx 0.056158765LaTeX: C_1=(-0.6)\times(-0.093597941)\approx 0.056158765

Accumulate the contributions, then apply the hidden activation derivative below.

\text{Sum}=C_1\approx 0.056158765LaTeX: \text{Sum}=C_1\approx 0.056158765
\delta_{h_3} = 0.056158765 \times 0.231380704 = 0.012994055LaTeX: \delta_{h_3} = 0.056158765 \times 0.231380704 = 0.012994055

Hidden 4 downstream contribution:

Calculate each outgoing path using the old outgoing weight and that output's delta.

Several output paths meet at this hidden neuron
Output 1
Output delta−0.093597941
Old outgoing weight0.2
Path contribution−0.018719588
Add every path contribution−0.018719588Then multiply by the hidden activation derivative.

Multiply within each path. Add the resulting contributions at the shared hidden neuron.

Contribution from output 1.

C_1=(0.2)\times(-0.093597941)\approx -0.018719588LaTeX: C_1=(0.2)\times(-0.093597941)\approx -0.018719588

Accumulate the contributions, then apply the hidden activation derivative below.

\text{Sum}=C_1\approx -0.018719588LaTeX: \text{Sum}=C_1\approx -0.018719588
\delta_{h_4} = -0.018719588 \times 0.22878424 = -0.004282747LaTeX: \delta_{h_4} = -0.018719588 \times 0.22878424 = -0.004282747

Step 5 input weights. Multiply the hidden delta by the corresponding raw input, not by the hidden output.

x1 → h1:

The calculation on x1 → h1
Sending input0.4x1
Receiving delta−0.009617203δ at h1
Connection gradient−0.003846881x1 → h1
Now update this weight
Old weight0.8
New weight0.801923441

Subtract η × gradient = −0.00192344. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{1,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,1}}=\delta_{h_1}x_1LaTeX: g_{1,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,1}}=\delta_{h_1}x_1
g_{1,1}^{(1)}=(-0.009617203)\times(0.4)\approx -0.003846881LaTeX: g_{1,1}^{(1)}=(-0.009617203)\times(0.4)\approx -0.003846881

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{1,1,\mathrm{new}}=0.8-0.5(-0.003846881)\approx 0.801923441LaTeX: W^{(1)}_{1,1,\mathrm{new}}=0.8-0.5(-0.003846881)\approx 0.801923441

x2 → h1:

The calculation on x2 → h1
Sending input0.3x2
Receiving delta−0.009617203δ at h1
Connection gradient−0.002885161x2 → h1
Now update this weight
Old weight−0.5
New weight−0.49855742

Subtract η × gradient = −0.00144258. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{1,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,2}}=\delta_{h_1}x_2LaTeX: g_{1,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,2}}=\delta_{h_1}x_2
g_{1,2}^{(1)}=(-0.009617203)\times(0.3)\approx -0.002885161LaTeX: g_{1,2}^{(1)}=(-0.009617203)\times(0.3)\approx -0.002885161

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{1,2,\mathrm{new}}=-0.5-0.5(-0.002885161)\approx -0.49855742LaTeX: W^{(1)}_{1,2,\mathrm{new}}=-0.5-0.5(-0.002885161)\approx -0.49855742

x3 → h1:

The calculation on x3 → h1
Sending input0.5x3
Receiving delta−0.009617203δ at h1
Connection gradient−0.004808601x3 → h1
Now update this weight
Old weight0.1
New weight0.102404301

Subtract η × gradient = −0.0024043. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{1,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,3}}=\delta_{h_1}x_3LaTeX: g_{1,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,3}}=\delta_{h_1}x_3
g_{1,3}^{(1)}=(-0.009617203)\times(0.5)\approx -0.004808601LaTeX: g_{1,3}^{(1)}=(-0.009617203)\times(0.5)\approx -0.004808601

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{1,3,\mathrm{new}}=0.1-0.5(-0.004808601)\approx 0.102404301LaTeX: W^{(1)}_{1,3,\mathrm{new}}=0.1-0.5(-0.004808601)\approx 0.102404301

x4 → h1:

The calculation on x4 → h1
Sending input−0.4x4
Receiving delta−0.009617203δ at h1
Connection gradient0.003846881x4 → h1
Now update this weight
Old weight−0.2
New weight−0.201923441

Subtract η × gradient = 0.00192344. This weight decreases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{1,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,4}}=\delta_{h_1}x_4LaTeX: g_{1,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,4}}=\delta_{h_1}x_4
g_{1,4}^{(1)}=(-0.009617203)\times(-0.4)\approx 0.003846881LaTeX: g_{1,4}^{(1)}=(-0.009617203)\times(-0.4)\approx 0.003846881

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{1,4,\mathrm{new}}=-0.2-0.5(0.003846881)\approx -0.201923441LaTeX: W^{(1)}_{1,4,\mathrm{new}}=-0.2-0.5(0.003846881)\approx -0.201923441

x1 → h2:

The calculation on x1 → h2
Sending input0.4x1
Receiving delta−0.014526308δ at h2
Connection gradient−0.005810523x1 → h2
Now update this weight
Old weight0.2
New weight0.202905262

Subtract η × gradient = −0.002905262. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{2,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,1}}=\delta_{h_2}x_1LaTeX: g_{2,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,1}}=\delta_{h_2}x_1
g_{2,1}^{(1)}=(-0.014526308)\times(0.4)\approx -0.005810523LaTeX: g_{2,1}^{(1)}=(-0.014526308)\times(0.4)\approx -0.005810523

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{2,1,\mathrm{new}}=0.2-0.5(-0.005810523)\approx 0.202905262LaTeX: W^{(1)}_{2,1,\mathrm{new}}=0.2-0.5(-0.005810523)\approx 0.202905262

x2 → h2:

The calculation on x2 → h2
Sending input0.3x2
Receiving delta−0.014526308δ at h2
Connection gradient−0.004357892x2 → h2
Now update this weight
Old weight0.4
New weight0.402178946

Subtract η × gradient = −0.002178946. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{2,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,2}}=\delta_{h_2}x_2LaTeX: g_{2,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,2}}=\delta_{h_2}x_2
g_{2,2}^{(1)}=(-0.014526308)\times(0.3)\approx -0.004357892LaTeX: g_{2,2}^{(1)}=(-0.014526308)\times(0.3)\approx -0.004357892

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{2,2,\mathrm{new}}=0.4-0.5(-0.004357892)\approx 0.402178946LaTeX: W^{(1)}_{2,2,\mathrm{new}}=0.4-0.5(-0.004357892)\approx 0.402178946

x3 → h2:

The calculation on x3 → h2
Sending input0.5x3
Receiving delta−0.014526308δ at h2
Connection gradient−0.007263154x3 → h2
Now update this weight
Old weight0.2
New weight0.203631577

Subtract η × gradient = −0.003631577. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{2,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,3}}=\delta_{h_2}x_3LaTeX: g_{2,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,3}}=\delta_{h_2}x_3
g_{2,3}^{(1)}=(-0.014526308)\times(0.5)\approx -0.007263154LaTeX: g_{2,3}^{(1)}=(-0.014526308)\times(0.5)\approx -0.007263154

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{2,3,\mathrm{new}}=0.2-0.5(-0.007263154)\approx 0.203631577LaTeX: W^{(1)}_{2,3,\mathrm{new}}=0.2-0.5(-0.007263154)\approx 0.203631577

x4 → h2:

The calculation on x4 → h2
Sending input−0.4x4
Receiving delta−0.014526308δ at h2
Connection gradient0.005810523x4 → h2
Now update this weight
Old weight0.5
New weight0.497094738

Subtract η × gradient = 0.002905262. This weight decreases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{2,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,4}}=\delta_{h_2}x_4LaTeX: g_{2,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,4}}=\delta_{h_2}x_4
g_{2,4}^{(1)}=(-0.014526308)\times(-0.4)\approx 0.005810523LaTeX: g_{2,4}^{(1)}=(-0.014526308)\times(-0.4)\approx 0.005810523

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{2,4,\mathrm{new}}=0.5-0.5(0.005810523)\approx 0.497094738LaTeX: W^{(1)}_{2,4,\mathrm{new}}=0.5-0.5(0.005810523)\approx 0.497094738

x1 → h3:

The calculation on x1 → h3
Sending input0.4x1
Receiving delta0.012994055δ at h3
Connection gradient0.005197622x1 → h3
Now update this weight
Old weight0.3
New weight0.297401189

Subtract η × gradient = 0.002598811. This weight decreases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{3,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,1}}=\delta_{h_3}x_1LaTeX: g_{3,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,1}}=\delta_{h_3}x_1
g_{3,1}^{(1)}=(0.012994055)\times(0.4)\approx 0.005197622LaTeX: g_{3,1}^{(1)}=(0.012994055)\times(0.4)\approx 0.005197622

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{3,1,\mathrm{new}}=0.3-0.5(0.005197622)\approx 0.297401189LaTeX: W^{(1)}_{3,1,\mathrm{new}}=0.3-0.5(0.005197622)\approx 0.297401189

x2 → h3:

The calculation on x2 → h3
Sending input0.3x2
Receiving delta0.012994055δ at h3
Connection gradient0.003898216x2 → h3
Now update this weight
Old weight0.3
New weight0.298050892

Subtract η × gradient = 0.001949108. This weight decreases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{3,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,2}}=\delta_{h_3}x_2LaTeX: g_{3,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,2}}=\delta_{h_3}x_2
g_{3,2}^{(1)}=(0.012994055)\times(0.3)\approx 0.003898216LaTeX: g_{3,2}^{(1)}=(0.012994055)\times(0.3)\approx 0.003898216

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{3,2,\mathrm{new}}=0.3-0.5(0.003898216)\approx 0.298050892LaTeX: W^{(1)}_{3,2,\mathrm{new}}=0.3-0.5(0.003898216)\approx 0.298050892

x3 → h3:

The calculation on x3 → h3
Sending input0.5x3
Receiving delta0.012994055δ at h3
Connection gradient0.006497027x3 → h3
Now update this weight
Old weight−0.1
New weight−0.103248514

Subtract η × gradient = 0.003248514. This weight decreases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{3,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,3}}=\delta_{h_3}x_3LaTeX: g_{3,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,3}}=\delta_{h_3}x_3
g_{3,3}^{(1)}=(0.012994055)\times(0.5)\approx 0.006497027LaTeX: g_{3,3}^{(1)}=(0.012994055)\times(0.5)\approx 0.006497027

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{3,3,\mathrm{new}}=-0.1-0.5(0.006497027)\approx -0.103248514LaTeX: W^{(1)}_{3,3,\mathrm{new}}=-0.1-0.5(0.006497027)\approx -0.103248514

x4 → h3:

The calculation on x4 → h3
Sending input−0.4x4
Receiving delta0.012994055δ at h3
Connection gradient−0.005197622x4 → h3
Now update this weight
Old weight0.5
New weight0.502598811

Subtract η × gradient = −0.002598811. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{3,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,4}}=\delta_{h_3}x_4LaTeX: g_{3,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,4}}=\delta_{h_3}x_4
g_{3,4}^{(1)}=(0.012994055)\times(-0.4)\approx -0.005197622LaTeX: g_{3,4}^{(1)}=(0.012994055)\times(-0.4)\approx -0.005197622

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{3,4,\mathrm{new}}=0.5-0.5(-0.005197622)\approx 0.502598811LaTeX: W^{(1)}_{3,4,\mathrm{new}}=0.5-0.5(-0.005197622)\approx 0.502598811

x1 → h4:

The calculation on x1 → h4
Sending input0.4x1
Receiving delta−0.004282747δ at h4
Connection gradient−0.001713099x1 → h4
Now update this weight
Old weight0.4
New weight0.400856549

Subtract η × gradient = −0.00085655. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{4,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,1}}=\delta_{h_4}x_1LaTeX: g_{4,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,1}}=\delta_{h_4}x_1
g_{4,1}^{(1)}=(-0.004282747)\times(0.4)\approx -0.001713099LaTeX: g_{4,1}^{(1)}=(-0.004282747)\times(0.4)\approx -0.001713099

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{4,1,\mathrm{new}}=0.4-0.5(-0.001713099)\approx 0.400856549LaTeX: W^{(1)}_{4,1,\mathrm{new}}=0.4-0.5(-0.001713099)\approx 0.400856549

x2 → h4:

The calculation on x2 → h4
Sending input0.3x2
Receiving delta−0.004282747δ at h4
Connection gradient−0.001284824x2 → h4
Now update this weight
Old weight0.2
New weight0.200642412

Subtract η × gradient = −0.000642412. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{4,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,2}}=\delta_{h_4}x_2LaTeX: g_{4,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,2}}=\delta_{h_4}x_2
g_{4,2}^{(1)}=(-0.004282747)\times(0.3)\approx -0.001284824LaTeX: g_{4,2}^{(1)}=(-0.004282747)\times(0.3)\approx -0.001284824

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{4,2,\mathrm{new}}=0.2-0.5(-0.001284824)\approx 0.200642412LaTeX: W^{(1)}_{4,2,\mathrm{new}}=0.2-0.5(-0.001284824)\approx 0.200642412

x3 → h4:

The calculation on x3 → h4
Sending input0.5x3
Receiving delta−0.004282747δ at h4
Connection gradient−0.002141373x3 → h4
Now update this weight
Old weight0.2
New weight0.201070687

Subtract η × gradient = −0.001070686. This weight increases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{4,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,3}}=\delta_{h_4}x_3LaTeX: g_{4,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,3}}=\delta_{h_4}x_3
g_{4,3}^{(1)}=(-0.004282747)\times(0.5)\approx -0.002141373LaTeX: g_{4,3}^{(1)}=(-0.004282747)\times(0.5)\approx -0.002141373

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{4,3,\mathrm{new}}=0.2-0.5(-0.002141373)\approx 0.201070687LaTeX: W^{(1)}_{4,3,\mathrm{new}}=0.2-0.5(-0.002141373)\approx 0.201070687

x4 → h4:

The calculation on x4 → h4
Sending input−0.4x4
Receiving delta−0.004282747δ at h4
Connection gradient0.001713099x4 → h4
Now update this weight
Old weight0.8
New weight0.799143451

Subtract η × gradient = 0.00085655. This weight decreases.

The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights.

The receiving neuron supplies the delta; the sending neuron supplies the input value.

g_{4,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,4}}=\delta_{h_4}x_4LaTeX: g_{4,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,4}}=\delta_{h_4}x_4
g_{4,4}^{(1)}=(-0.004282747)\times(-0.4)\approx 0.001713099LaTeX: g_{4,4}^{(1)}=(-0.004282747)\times(-0.4)\approx 0.001713099

Apply the learning rate to this gradient and subtract the result from the old weight.

W^{(1)}_{4,4,\mathrm{new}}=0.8-0.5(0.001713099)\approx 0.799143451LaTeX: W^{(1)}_{4,4,\mathrm{new}}=0.8-0.5(0.001713099)\approx 0.799143451

Step 6 bias updates. A bias receives a constant input 1, so its gradient equals the corresponding delta. Use the same old forward pass and learning rate as the weight updates.

b_{h_1,\mathrm{new}}=0.6-0.5(-0.009617203)=0.604808601LaTeX: b_{h_1,\mathrm{new}}=0.6-0.5(-0.009617203)=0.604808601
b_{h_2,\mathrm{new}}=0.6-0.5(-0.014526308)=0.607263154LaTeX: b_{h_2,\mathrm{new}}=0.6-0.5(-0.014526308)=0.607263154
b_{h_3,\mathrm{new}}=0.6-0.5(0.012994055)=0.593502972LaTeX: b_{h_3,\mathrm{new}}=0.6-0.5(0.012994055)=0.593502972
b_{h_4,\mathrm{new}}=0.6-0.5(-0.004282747)=0.602141373LaTeX: b_{h_4,\mathrm{new}}=0.6-0.5(-0.004282747)=0.602141373
b_{o_1,\mathrm{new}}=0.2-0.5(-0.093597941)=0.246798971LaTeX: b_{o_1,\mathrm{new}}=0.2-0.5(-0.093597941)=0.246798971

These are independent per-neuron biases. If the question fixes biases, keep them fixed. If it specifies a single tied bias, add all deltas sharing that bias before updating it.

Required result: output 0.683621675, MSE 0.046819580, h₁ → output new weight 0.533271705. If instead each printed row is assumed to receive inputs, the conditional alternative is output 0.681090691, MSE 0.047921285 and updated weight 0.534478309. Explain the arrow convention rather than mixing these results.