Deep Learning · TEE · section

R24 Q4a

Re examination February 2024 fully solved

Question

Question (10 marks). Train a perceptron for 2 epochs with η = 1, initial weights and bias 0, threshold 0. Samples in order: [1,1,1,1] → 1; [−1,1,−1,−1] → 1; [1,1,1,−1] → −1; [1,−1,−1,1] → −1. Bias input is 1.

Answer — step by step

Given

Train a perceptron for 2 epochs with η = 1, initial weights and bias 0, threshold 0.

Samples in order: [1,1,1,1] → 1.

[−1,1,−1,−1] → 1.

[1,1,1,−1] → −1.

[1,−1,−1,1] → −1.

Bias input is 1.

To find

Process samples in order for the stated epochs and report the final weights and bias.

Formula — which one and why

Target-based perceptron correction. This worked convention changes parameters only on a wrong prediction. Use the target itself as the signed multiplier; it differs from the binary error rule.

t is the bipolar target; η is the learning rate. Output is +1 above zero, −1 below zero and 0 at exactly zero.

\Delta w=\eta t x,\quad\Delta b=\eta t\quad\text{on a mismatch}LaTeX: \Delta w=\eta t x,\quad\Delta b=\eta t\quad\text{on a mismatch}

Yaad rakhne ka cue: Is convention mein galat hua toh target × input se correction.

Solution — step by step

State the threshold rule. Use output +1 if z>0, −1 if z<0, and 0 at z = 0. On a mismatch use Δw = η tx and Δb = ηt; on a match make no update. η = 1. A different rule at zero can change the learning trajectory. Write your convention explicitly.

Epoch 1, sample 1

Step 1: copy this sample and target.

x = [1, 1, 1, 1], target = 1.

Step 2: calculate the current net.

z = (0 × 1) + (0 × 1) + (0 × 1) + (0 × 1) + (0) = 0

Step 3: apply the threshold.

output = 0.

Step 4: calculate the update multiplier.

Update multiplier = 1

Step 5: calculate all weight changes.

Δw = 1 × [1, 1, 1, 1] = [1, 1, 1, 1]

Step 6: calculate the bias change.

Δb = 1.

Step 7: carry the updated parameters forward.

New w = [1, 1, 1, 1], new b = 1.

Epoch 1, sample 2

Step 1: copy this sample and target.

x = [-1, 1, -1, -1], target = 1.

Step 2: calculate the current net.

z = (1 × -1) + (1 × 1) + (1 × -1) + (1 × -1) + (1) = -1

Step 3: apply the threshold.

output = -1.

Step 4: calculate the update multiplier.

Update multiplier = 1

Step 5: calculate all weight changes.

Δw = 1 × [-1, 1, -1, -1] = [-1, 1, -1, -1]

Step 6: calculate the bias change.

Δb = 1.

Step 7: carry the updated parameters forward.

New w = [0, 2, 0, 0], new b = 2.

Epoch 1, sample 3

Step 1: copy this sample and target.

x = [1, 1, 1, -1], target = -1.

Step 2: calculate the current net.

z = (0 × 1) + (2 × 1) + (0 × 1) + (0 × -1) + (2) = 4

Step 3: apply the threshold.

output = 1.

Step 4: calculate the update multiplier.

Update multiplier = -1

Step 5: calculate all weight changes.

Δw = -1 × [1, 1, 1, -1] = [-1, -1, -1, 1]

Step 6: calculate the bias change.

Δb = -1.

Step 7: carry the updated parameters forward.

New w = [-1, 1, -1, 1], new b = 1.

Epoch 1, sample 4

Step 1: copy this sample and target.

x = [1, -1, -1, 1], target = -1.

Step 2: calculate the current net.

z = (-1 × 1) + (1 × -1) + (-1 × -1) + (1 × 1) + (1) = 1

Step 3: apply the threshold.

output = 1.

Step 4: calculate the update multiplier.

Update multiplier = -1

Step 5: calculate all weight changes.

Δw = -1 × [1, -1, -1, 1] = [-1, 1, 1, -1]

Step 6: calculate the bias change.

Δb = -1.

Step 7: carry the updated parameters forward.

New w = [-2, 2, 0, 0], new b = 0.

Epoch 2, sample 1

Step 1: copy this sample and target.

x = [1, 1, 1, 1], target = 1.

Step 2: calculate the current net.

z = (-2 × 1) + (2 × 1) + (0 × 1) + (0 × 1) + (0) = 0

Step 3: apply the threshold.

output = 0.

Step 4: calculate the update multiplier.

Update multiplier = 1

Step 5: calculate all weight changes.

Δw = 1 × [1, 1, 1, 1] = [1, 1, 1, 1]

Step 6: calculate the bias change.

Δb = 1.

Step 7: carry the updated parameters forward.

New w = [-1, 3, 1, 1], new b = 1.

Epoch 2, sample 2

Step 1: copy this sample and target.

x = [-1, 1, -1, -1], target = 1.

Step 2: calculate the current net.

z = (-1 × -1) + (3 × 1) + (1 × -1) + (1 × -1) + (1) = 3

Step 3: apply the threshold.

output = 1.

Step 4: calculate the update multiplier.

Update multiplier = 0

Step 5: calculate all weight changes.

Δw = 0 × [-1, 1, -1, -1] = [0, 0, 0, 0]

Step 6: calculate the bias change.

Δb = 0.

Step 7: carry the updated parameters forward.

New w = [-1, 3, 1, 1], new b = 1.

Epoch 2, sample 3

Step 1: copy this sample and target.

x = [1, 1, 1, -1], target = -1.

Step 2: calculate the current net.

z = (-1 × 1) + (3 × 1) + (1 × 1) + (1 × -1) + (1) = 3

Step 3: apply the threshold.

output = 1.

Step 4: calculate the update multiplier.

Update multiplier = -1

Step 5: calculate all weight changes.

Δw = -1 × [1, 1, 1, -1] = [-1, -1, -1, 1]

Step 6: calculate the bias change.

Δb = -1.

Step 7: carry the updated parameters forward.

New w = [-2, 2, 0, 2], new b = 0.

Epoch 2, sample 4

Step 1: copy this sample and target.

x = [1, -1, -1, 1], target = -1.

Step 2: calculate the current net.

z = (-2 × 1) + (2 × -1) + (0 × -1) + (2 × 1) + (0) = -2

Step 3: apply the threshold.

output = -1.

Step 4: calculate the update multiplier.

Update multiplier = 0

Step 5: calculate all weight changes.

Δw = 0 × [1, -1, -1, 1] = [0, 0, 0, 0]

Step 6: calculate the bias change.

Δb = 0.

Step 7: carry the updated parameters forward.

New w = [-2, 2, 0, 2], new b = 0.

Final after exactly the requested epochs: w = [-2, 2, 0, 2], b = 0. A finite number of requested epochs does not itself prove convergence or correct classification of every sample.