# R24 Q2a Subject: Deep Learning Exam: TEE Reader: https://deep-learning-study-room.vercel.app/#tee/read/14/tee-c14-s4 Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c14-s4/ ## Question **Question (10 marks).** A 4 → 4 → 1 sigmoid network has input [.4,.3,.5,−.4]. The printed W₁ is shown below, W₂ = [.5,.7,−.6,.2]; hidden bias 0.6, output bias 0.2, target 0.9, η = .5. Compute mean squared error and update the connection h₁ → output. In the graph θ ij connects input i to hidden j. ![Given matrices for this question](https://deep-learning-study-room.vercel.app/assets/figures/given_28edc44005.svg) Diagram: Given matrices for this question. Labels: GIVEN VALUES — read each matrix row from left to right | W₁ | 0.8 | 0.2 | 0.3 | 0.4 | −0.5 | 0.4 | 0.3 | 0.2 | 0.1 | 0.2 | −0.1 | 0.2 | −0.2 | 0.5 | 0.5 | 0.8 Given matrices for this question Tap the diagram to enlarge. ![R24 Q2a receiving-neuron rows after transposing the source convention](https://deep-learning-study-room.vercel.app/assets/figures/net_r24.svg) Diagram: R24 Q2a receiving-neuron rows after transposing the source convention. Labels: Read each receiving neuron as one row of weights | INPUT | HIDDEN | OUTPUT | x1=0.4 | x2=0.3 | x3=0.5 | x4=-0.4 | h1 | h2 | h3 | h4 | o1 | target 0.9 | h1 incoming: x1 × 0.8 + x2 × -0.5 + x3 × 0.1 + x4 × -0.2; bias 0.6 | h2 incoming: x1 × 0.2 + x2 × 0.4 + x3 × 0.2 + x4 × 0.5; bias 0.6 | h3 incoming: x1 × 0.3 + x2 × 0.3 + x3 × -0.1 + x4 × 0.5; bias 0.6 | h4 incoming: x1 × 0.4 + x2 × 0.2 + x3 × 0.2 + x4 × 0.8; bias 0.6 | o1 incoming: h1 × 0.5 + h2 × 0.7 + h3 × -0.6 + h4 × 0.2; bias 0.2 | Use these OLD connections for all gradients in the same pass. R24 Q2a receiving-neuron rows after transposing the source convention Tap the diagram to enlarge. ## Answer — step by step ### Given A 4 → 4 → 1 sigmoid network has input [.4,.3,.5,−.4]. The printed W₁ is printed with the question, W₂ = [.5,.7,−.6,.2]. hidden bias 0.6, output bias 0.2, target 0.9, η = .5. ### To find Find all preactivations, hidden activations and final outputs. Calculate MSE and its output derivatives. Find the sigmoid output and, if requested, its derivative. Find the requested deltas and parameter gradients, then update the specified parameters. ### Formula — which one and why **Forward propagation.** Evaluate one layer completely before supplying its activated outputs to the next layer. Each layer has its own weights and biases. x = input vector; h = hidden outputs; W₁,b₁ = hidden parameters; W₂,b₂ = output parameters; f₁,f₂ = the stated activations. $$ z_h=W_1x+b_1 $$ $$ h=f_1(z_h) $$ $$ z_o=W_2h+b_2 $$ $$ o=f_2(z_o) $$ **Yaad rakhne ka cue:** Pehli layer ka activated output hi agli layer ka input hai. **Mean squared loss.** Average the squared scalar prediction errors. The derivative retains the factor 2/n; n must count the outputs included in this mean. n = number of scalar prediction errors in the stated mean; oⱼ = prediction; yⱼ = target. $$ L=\frac1n\sum_j(o_j-y_j)^2 $$ $$ \frac{\partial L}{\partial o_j}=\frac2n(o_j-y_j) $$ **Yaad rakhne ka cue:** MSE mein derivative 2/n times error; half SSE ka rule blindly mat lagao. **Sigmoid and its slope.** Use sigmoid when the neuron is specified to squash its preactivation into (0,1). Once its output is known, its derivative needs only output × (1−output). z = preactivation; p = sigmoid output; e is the exponential base, not a prediction error in this formula. $$ p=\frac{1}{1+e^{-z}} $$ $$ \frac{dp}{dz}=p(1-p) $$ **Yaad rakhne ka cue:** Sigmoid output p; uska slope p × (1−p). **Backpropagation.** The loss tells us how each output must change. Multiply by that neuron’s activation slope to obtain its delta. A weight gradient is receiving delta × sending value. Hidden deltas must include every outgoing path. δ (delta) = derivative of loss with respect to a neuron’s preactivation; f′ = activation derivative; aᵢ = sending activation; Wⱼᵢ = connection weight. Use the exact loss scaling in the question. $$ \delta_{o_j}=\frac{\partial L}{\partial o_j}f_o\prime(z_{o_j}) $$ $$ \delta_{h_i}=f_h\prime(z_{h_i})\sum_jW_{ji}^{(2)}\delta_{o_j} $$ $$ \frac{\partial L}{\partial W_{ji}}=\delta_ja_i $$ $$ \frac{\partial L}{\partial b_j}=\delta_j $$ $$ W_{ji,\mathrm{new}}=W_{ji}-\eta\frac{\partial L}{\partial W_{ji}} $$ **Yaad rakhne ka cue:** Receiving delta × sending value = us connection ka gradient. ### Solution — step by step **Orientation:** the source rows list sending inputs, so use each COLUMN to form a receiving hidden neuron. In our usual column-input convention this means using the transpose of printed W₁. The loss is one-output MSE = (o−0.9)², so its derivative includes 2. **Step 1 map the arrows.** Each row of W₁ receives all inputs; each row of W₂ receives all hidden outputs. Bias is added once per receiving neuron. All gradients below use the OLD weights from this same forward pass. **Hidden neuron 1:** $$ z_{h_1} = (0.8 \times 0.4) + (-0.5 \times 0.3) + (0.1 \times 0.5) + (-0.2 \times -0.4) + (0.6) = 0.9 $$ $$ h_{1}=\frac{1}{1+e^{-(0.9)}}=0.710949503 $$ **Hidden neuron 2:** $$ z_{h_2} = (0.2 \times 0.4) + (0.4 \times 0.3) + (0.2 \times 0.5) + (0.5 \times -0.4) + (0.6) = 0.7 $$ $$ h_{2}=\frac{1}{1+e^{-(0.7)}}=0.668187772 $$ **Hidden neuron 3:** $$ z_{h_3} = (0.3 \times 0.4) + (0.3 \times 0.3) + (-0.1 \times 0.5) + (0.5 \times -0.4) + (0.6) = 0.56 $$ $$ h_{3}=\frac{1}{1+e^{-(0.56)}}=0.63645254 $$ **Hidden neuron 4:** $$ z_{h_4} = (0.4 \times 0.4) + (0.2 \times 0.3) + (0.2 \times 0.5) + (0.8 \times -0.4) + (0.6) = 0.6 $$ $$ h_{4}=\frac{1}{1+e^{-(0.6)}}=0.645656306 $$ **Output neuron 1:** $$ \begin{aligned}z_{o_1}={}&0.5(0.710949503)+0.7(0.668187772)\\&-0.6(0.63645254)+0.2(0.645656306)+0.2\\={}&0.770465929\end{aligned} $$ $$ o_{1}=\frac{1}{1+e^{-(0.770465929)}}=0.683621675 $$ **Step 2 loss convention.** Use the explicitly requested one-output MSE, not half squared loss. Prediction minus target = [-0.216378325]. Squared errors = [0.04681958]. Half-sum squared loss = 0.02340979; mean squared error = 0.04681958. These are different unless the number of output components makes the scale coincide. **Output delta 1:** $$ \delta_{o_1} = 2 \times (0.683621675-0.9) \times 0.216283081 = -0.093597941 $$ The derivative is o(1−o). **Step 3 output weights.** Gradient = receiving delta × sending activation. An output bias gradient equals its delta. **h1 → o1:** The calculation on h1 → o1 Sending activation**0.710949503**h1 × Receiving delta**−0.093597941**δ at o1 = Connection gradient**−0.06654341**h1 → o1 Now update this weight Old weight**0.5** → New weight**0.533271705** Subtract η × gradient = **−0.033271705**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies its activation. $$ g_{1,1}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,1}}=\delta_{o_1}h_1 $$ $$ g_{1,1}^{(2)}=(-0.093597941)\times(0.710949503)\approx -0.06654341 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(2)}_{1,1,\mathrm{new}}=0.5-0.5(-0.06654341)\approx 0.533271705 $$ **h2 → o1:** The calculation on h2 → o1 Sending activation**0.668187772**h2 × Receiving delta**−0.093597941**δ at o1 = Connection gradient**−0.062541**h2 → o1 Now update this weight Old weight**0.7** → New weight**0.7312705** Subtract η × gradient = **−0.0312705**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies its activation. $$ g_{1,2}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,2}}=\delta_{o_1}h_2 $$ $$ g_{1,2}^{(2)}=(-0.093597941)\times(0.668187772)\approx -0.062541 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(2)}_{1,2,\mathrm{new}}=0.7-0.5(-0.062541)\approx 0.7312705 $$ **h3 → o1:** The calculation on h3 → o1 Sending activation**0.63645254**h3 × Receiving delta**−0.093597941**δ at o1 = Connection gradient**−0.059570648**h3 → o1 Now update this weight Old weight**−0.6** → New weight**−0.570214676** Subtract η × gradient = **−0.029785324**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies its activation. $$ g_{1,3}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,3}}=\delta_{o_1}h_3 $$ $$ g_{1,3}^{(2)}=(-0.093597941)\times(0.63645254)\approx -0.059570648 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(2)}_{1,3,\mathrm{new}}=-0.6-0.5(-0.059570648)\approx -0.570214676 $$ **h4 → o1:** The calculation on h4 → o1 Sending activation**0.645656306**h4 × Receiving delta**−0.093597941**δ at o1 = Connection gradient**−0.060432101**h4 → o1 Now update this weight Old weight**0.2** → New weight**0.230216051** Subtract η × gradient = **−0.03021605**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies its activation. $$ g_{1,4}^{(2)}=\frac{\partial L}{\partial W^{(2)}_{1,4}}=\delta_{o_1}h_4 $$ $$ g_{1,4}^{(2)}=(-0.093597941)\times(0.645656306)\approx -0.060432101 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(2)}_{1,4,\mathrm{new}}=0.2-0.5(-0.060432101)\approx 0.230216051 $$ **Step 4 hidden deltas.** A hidden unit can influence EVERY connected output, so add those paths BEFORE multiplying by the hidden activation derivative. **Hidden 1 downstream contribution:** Calculate each outgoing path using the **old** outgoing weight and that output's delta. Several output paths meet at this hidden neuron Output 1 Output delta**−0.093597941** × Old outgoing weight**0.5** = Path contribution**−0.04679897** ↓ Add every path contribution**−0.046798971**Then multiply by the hidden activation derivative. Multiply within each path. Add the resulting contributions at the shared hidden neuron. **Contribution from output 1.** $$ C_1=(0.5)\times(-0.093597941)\approx -0.04679897 $$ **Accumulate the contributions, then apply the hidden activation derivative below.** $$ \text{Sum}=C_1\approx -0.046798971 $$ $$ \delta_{h_1} = -0.046798971 \times 0.205500307 = -0.009617203 $$ **Hidden 2 downstream contribution:** Calculate each outgoing path using the **old** outgoing weight and that output's delta. Several output paths meet at this hidden neuron Output 1 Output delta**−0.093597941** × Old outgoing weight**0.7** = Path contribution**−0.065518559** ↓ Add every path contribution**−0.065518559**Then multiply by the hidden activation derivative. Multiply within each path. Add the resulting contributions at the shared hidden neuron. **Contribution from output 1.** $$ C_1=(0.7)\times(-0.093597941)\approx -0.065518559 $$ **Accumulate the contributions, then apply the hidden activation derivative below.** $$ \text{Sum}=C_1\approx -0.065518559 $$ $$ \delta_{h_2} = -0.065518559 \times 0.221712873 = -0.014526308 $$ **Hidden 3 downstream contribution:** Calculate each outgoing path using the **old** outgoing weight and that output's delta. Several output paths meet at this hidden neuron Output 1 Output delta**−0.093597941** × Old outgoing weight**−0.6** = Path contribution**0.056158765** ↓ Add every path contribution**0.056158765**Then multiply by the hidden activation derivative. Multiply within each path. Add the resulting contributions at the shared hidden neuron. **Contribution from output 1.** $$ C_1=(-0.6)\times(-0.093597941)\approx 0.056158765 $$ **Accumulate the contributions, then apply the hidden activation derivative below.** $$ \text{Sum}=C_1\approx 0.056158765 $$ $$ \delta_{h_3} = 0.056158765 \times 0.231380704 = 0.012994055 $$ **Hidden 4 downstream contribution:** Calculate each outgoing path using the **old** outgoing weight and that output's delta. Several output paths meet at this hidden neuron Output 1 Output delta**−0.093597941** × Old outgoing weight**0.2** = Path contribution**−0.018719588** ↓ Add every path contribution**−0.018719588**Then multiply by the hidden activation derivative. Multiply within each path. Add the resulting contributions at the shared hidden neuron. **Contribution from output 1.** $$ C_1=(0.2)\times(-0.093597941)\approx -0.018719588 $$ **Accumulate the contributions, then apply the hidden activation derivative below.** $$ \text{Sum}=C_1\approx -0.018719588 $$ $$ \delta_{h_4} = -0.018719588 \times 0.22878424 = -0.004282747 $$ **Step 5 input weights.** Multiply the hidden delta by the corresponding raw input, not by the hidden output. **x1 → h1:** The calculation on x1 → h1 Sending input**0.4**x1 × Receiving delta**−0.009617203**δ at h1 = Connection gradient**−0.003846881**x1 → h1 Now update this weight Old weight**0.8** → New weight**0.801923441** Subtract η × gradient = **−0.00192344**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{1,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,1}}=\delta_{h_1}x_1 $$ $$ g_{1,1}^{(1)}=(-0.009617203)\times(0.4)\approx -0.003846881 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{1,1,\mathrm{new}}=0.8-0.5(-0.003846881)\approx 0.801923441 $$ **x2 → h1:** The calculation on x2 → h1 Sending input**0.3**x2 × Receiving delta**−0.009617203**δ at h1 = Connection gradient**−0.002885161**x2 → h1 Now update this weight Old weight**−0.5** → New weight**−0.49855742** Subtract η × gradient = **−0.00144258**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{1,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,2}}=\delta_{h_1}x_2 $$ $$ g_{1,2}^{(1)}=(-0.009617203)\times(0.3)\approx -0.002885161 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{1,2,\mathrm{new}}=-0.5-0.5(-0.002885161)\approx -0.49855742 $$ **x3 → h1:** The calculation on x3 → h1 Sending input**0.5**x3 × Receiving delta**−0.009617203**δ at h1 = Connection gradient**−0.004808601**x3 → h1 Now update this weight Old weight**0.1** → New weight**0.102404301** Subtract η × gradient = **−0.0024043**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{1,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,3}}=\delta_{h_1}x_3 $$ $$ g_{1,3}^{(1)}=(-0.009617203)\times(0.5)\approx -0.004808601 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{1,3,\mathrm{new}}=0.1-0.5(-0.004808601)\approx 0.102404301 $$ **x4 → h1:** The calculation on x4 → h1 Sending input**−0.4**x4 × Receiving delta**−0.009617203**δ at h1 = Connection gradient**0.003846881**x4 → h1 Now update this weight Old weight**−0.2** → New weight**−0.201923441** Subtract η × gradient = **0.00192344**. This weight **decreases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{1,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{1,4}}=\delta_{h_1}x_4 $$ $$ g_{1,4}^{(1)}=(-0.009617203)\times(-0.4)\approx 0.003846881 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{1,4,\mathrm{new}}=-0.2-0.5(0.003846881)\approx -0.201923441 $$ **x1 → h2:** The calculation on x1 → h2 Sending input**0.4**x1 × Receiving delta**−0.014526308**δ at h2 = Connection gradient**−0.005810523**x1 → h2 Now update this weight Old weight**0.2** → New weight**0.202905262** Subtract η × gradient = **−0.002905262**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{2,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,1}}=\delta_{h_2}x_1 $$ $$ g_{2,1}^{(1)}=(-0.014526308)\times(0.4)\approx -0.005810523 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{2,1,\mathrm{new}}=0.2-0.5(-0.005810523)\approx 0.202905262 $$ **x2 → h2:** The calculation on x2 → h2 Sending input**0.3**x2 × Receiving delta**−0.014526308**δ at h2 = Connection gradient**−0.004357892**x2 → h2 Now update this weight Old weight**0.4** → New weight**0.402178946** Subtract η × gradient = **−0.002178946**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{2,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,2}}=\delta_{h_2}x_2 $$ $$ g_{2,2}^{(1)}=(-0.014526308)\times(0.3)\approx -0.004357892 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{2,2,\mathrm{new}}=0.4-0.5(-0.004357892)\approx 0.402178946 $$ **x3 → h2:** The calculation on x3 → h2 Sending input**0.5**x3 × Receiving delta**−0.014526308**δ at h2 = Connection gradient**−0.007263154**x3 → h2 Now update this weight Old weight**0.2** → New weight**0.203631577** Subtract η × gradient = **−0.003631577**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{2,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,3}}=\delta_{h_2}x_3 $$ $$ g_{2,3}^{(1)}=(-0.014526308)\times(0.5)\approx -0.007263154 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{2,3,\mathrm{new}}=0.2-0.5(-0.007263154)\approx 0.203631577 $$ **x4 → h2:** The calculation on x4 → h2 Sending input**−0.4**x4 × Receiving delta**−0.014526308**δ at h2 = Connection gradient**0.005810523**x4 → h2 Now update this weight Old weight**0.5** → New weight**0.497094738** Subtract η × gradient = **0.002905262**. This weight **decreases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{2,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{2,4}}=\delta_{h_2}x_4 $$ $$ g_{2,4}^{(1)}=(-0.014526308)\times(-0.4)\approx 0.005810523 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{2,4,\mathrm{new}}=0.5-0.5(0.005810523)\approx 0.497094738 $$ **x1 → h3:** The calculation on x1 → h3 Sending input**0.4**x1 × Receiving delta**0.012994055**δ at h3 = Connection gradient**0.005197622**x1 → h3 Now update this weight Old weight**0.3** → New weight**0.297401189** Subtract η × gradient = **0.002598811**. This weight **decreases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{3,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,1}}=\delta_{h_3}x_1 $$ $$ g_{3,1}^{(1)}=(0.012994055)\times(0.4)\approx 0.005197622 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{3,1,\mathrm{new}}=0.3-0.5(0.005197622)\approx 0.297401189 $$ **x2 → h3:** The calculation on x2 → h3 Sending input**0.3**x2 × Receiving delta**0.012994055**δ at h3 = Connection gradient**0.003898216**x2 → h3 Now update this weight Old weight**0.3** → New weight**0.298050892** Subtract η × gradient = **0.001949108**. This weight **decreases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{3,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,2}}=\delta_{h_3}x_2 $$ $$ g_{3,2}^{(1)}=(0.012994055)\times(0.3)\approx 0.003898216 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{3,2,\mathrm{new}}=0.3-0.5(0.003898216)\approx 0.298050892 $$ **x3 → h3:** The calculation on x3 → h3 Sending input**0.5**x3 × Receiving delta**0.012994055**δ at h3 = Connection gradient**0.006497027**x3 → h3 Now update this weight Old weight**−0.1** → New weight**−0.103248514** Subtract η × gradient = **0.003248514**. This weight **decreases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{3,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,3}}=\delta_{h_3}x_3 $$ $$ g_{3,3}^{(1)}=(0.012994055)\times(0.5)\approx 0.006497027 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{3,3,\mathrm{new}}=-0.1-0.5(0.006497027)\approx -0.103248514 $$ **x4 → h3:** The calculation on x4 → h3 Sending input**−0.4**x4 × Receiving delta**0.012994055**δ at h3 = Connection gradient**−0.005197622**x4 → h3 Now update this weight Old weight**0.5** → New weight**0.502598811** Subtract η × gradient = **−0.002598811**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{3,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{3,4}}=\delta_{h_3}x_4 $$ $$ g_{3,4}^{(1)}=(0.012994055)\times(-0.4)\approx -0.005197622 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{3,4,\mathrm{new}}=0.5-0.5(-0.005197622)\approx 0.502598811 $$ **x1 → h4:** The calculation on x1 → h4 Sending input**0.4**x1 × Receiving delta**−0.004282747**δ at h4 = Connection gradient**−0.001713099**x1 → h4 Now update this weight Old weight**0.4** → New weight**0.400856549** Subtract η × gradient = **−0.00085655**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{4,1}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,1}}=\delta_{h_4}x_1 $$ $$ g_{4,1}^{(1)}=(-0.004282747)\times(0.4)\approx -0.001713099 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{4,1,\mathrm{new}}=0.4-0.5(-0.001713099)\approx 0.400856549 $$ **x2 → h4:** The calculation on x2 → h4 Sending input**0.3**x2 × Receiving delta**−0.004282747**δ at h4 = Connection gradient**−0.001284824**x2 → h4 Now update this weight Old weight**0.2** → New weight**0.200642412** Subtract η × gradient = **−0.000642412**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{4,2}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,2}}=\delta_{h_4}x_2 $$ $$ g_{4,2}^{(1)}=(-0.004282747)\times(0.3)\approx -0.001284824 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{4,2,\mathrm{new}}=0.2-0.5(-0.001284824)\approx 0.200642412 $$ **x3 → h4:** The calculation on x3 → h4 Sending input**0.5**x3 × Receiving delta**−0.004282747**δ at h4 = Connection gradient**−0.002141373**x3 → h4 Now update this weight Old weight**0.2** → New weight**0.201070687** Subtract η × gradient = **−0.001070686**. This weight **increases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{4,3}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,3}}=\delta_{h_4}x_3 $$ $$ g_{4,3}^{(1)}=(-0.004282747)\times(0.5)\approx -0.002141373 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{4,3,\mathrm{new}}=0.2-0.5(-0.002141373)\approx 0.201070687 $$ **x4 → h4:** The calculation on x4 → h4 Sending input**−0.4**x4 × Receiving delta**−0.004282747**δ at h4 = Connection gradient**0.001713099**x4 → h4 Now update this weight Old weight**0.8** → New weight**0.799143451** Subtract η × gradient = **0.00085655**. This weight **decreases**. The sending value and receiving delta have different roles. All deltas still use the original forward-pass weights. The receiving neuron supplies the delta; the sending neuron supplies the input value. $$ g_{4,4}^{(1)}=\frac{\partial L}{\partial W^{(1)}_{4,4}}=\delta_{h_4}x_4 $$ $$ g_{4,4}^{(1)}=(-0.004282747)\times(-0.4)\approx 0.001713099 $$ Apply the learning rate to this gradient and subtract the result from the old weight. $$ W^{(1)}_{4,4,\mathrm{new}}=0.8-0.5(0.001713099)\approx 0.799143451 $$ **Step 6 bias updates.** A bias receives a constant input 1, so its gradient equals the corresponding delta. Use the same old forward pass and learning rate as the weight updates. $$ b_{h_1,\mathrm{new}}=0.6-0.5(-0.009617203)=0.604808601 $$ $$ b_{h_2,\mathrm{new}}=0.6-0.5(-0.014526308)=0.607263154 $$ $$ b_{h_3,\mathrm{new}}=0.6-0.5(0.012994055)=0.593502972 $$ $$ b_{h_4,\mathrm{new}}=0.6-0.5(-0.004282747)=0.602141373 $$ $$ b_{o_1,\mathrm{new}}=0.2-0.5(-0.093597941)=0.246798971 $$ These are independent per-neuron biases. If the question fixes biases, keep them fixed. If it specifies a single tied bias, add all deltas sharing that bias before updating it. **Required result:** output **0.683621675**, MSE **0.046819580**, h₁ → output new weight **0.533271705**. If instead each printed row is assumed to receive inputs, the conditional alternative is output 0.681090691, MSE 0.047921285 and updated weight 0.534478309. Explain the arrow convention rather than mixing these results.