Deep Learning · TEE · section

Drill 11 One perceptron learning update

Numerical practice from direct to mixed

Question

Question. Binary perceptron, w = [0,0], b = 0, x = [1,1], target 1, η = 0.5. Use output 1 if z>0 else 0. Perform one update.

Answer — step by step

Given

w = [0,0]; b = 0; x = [1,1]; target t = 1.

Learning rate η = 0.5.

Output 1 only when z > 0; otherwise output 0.

To find

Perform one update and report new weights and bias.

Formula — which one and why

Perceptron learning rule. Evaluate the stated threshold, compare the predicted class with its target, then correct weights and bias. The next sample uses the newly updated parameters.

t = target; o = thresholded output; η = learning rate; xᵢ = input i. Binary and bipolar output conventions must follow the question.

z=\sum_iw_ix_i+bLaTeX: z=\sum_iw_ix_i+b
w_i^{\mathrm{new}}=w_i+\eta(t-o)x_iLaTeX: w_i^{\mathrm{new}}=w_i+\eta(t-o)x_i
b^{\mathrm{new}}=b+\eta(t-o)LaTeX: b^{\mathrm{new}}=b+\eta(t-o)

Yaad rakhne ka cue: Prediction nikalo; target−prediction se correction nikalo.

Solution — step by step

Meaning and formula

A perceptron first computes a weighted sum z and then applies the exact threshold rule in the question. This question uses strictly greater than zero, so z = 0 produces output 0.

z=w_1x_1+w_2x_2+b,\qquad o=\begin{cases}1&z>0\\0&z\le0\end{cases}LaTeX: z=w_1x_1+w_2x_2+b,\qquad o=\begin{cases}1&z>0\\0&z\le0\end{cases}

The perceptron learning rule uses target minus output, written e below. This is a discrete perceptron rule, not the sigmoid backpropagation derivative.

e=t-o,\qquad w_{j,\mathrm{new}}=w_j+\eta e x_j,\qquad b_{\mathrm{new}}=b+\eta eLaTeX: e=t-o,\qquad w_{j,\mathrm{new}}=w_j+\eta e x_j,\qquad b_{\mathrm{new}}=b+\eta e

Step 1: obtain the old output

z=0\times1+0\times1+0=0,\qquad o=0LaTeX: z=0\times1+0\times1+0=0,\qquad o=0

Step 2: compare it with the target

e=1-0=1LaTeX: e=1-0=1

Step 3: update each weight separately

w_{1,\mathrm{new}}=0+0.5\times1\times1=0.5LaTeX: w_{1,\mathrm{new}}=0+0.5\times1\times1=0.5
w_{2,\mathrm{new}}=0+0.5\times1\times1=0.5LaTeX: w_{2,\mathrm{new}}=0+0.5\times1\times1=0.5
b_{\mathrm{new}}=0+0.5\times1=0.5LaTeX: b_{\mathrm{new}}=0+0.5\times1=0.5

Step 4: interpret the updated prediction

z_{\mathrm{new}}=0.5\times1+0.5\times1+0.5=1.5LaTeX: z_{\mathrm{new}}=0.5\times1+0.5\times1+0.5=1.5

Since 1.5 > 0, the new output is 1. Final parameters: w = [0.5, 0.5], b = 0.5.