Deep Learning · TEE · section

Perceptrons and the threshold

Neural-network foundations

A perceptron is a simple neuron used for classification with a threshold decision. If a question says output 1 for z ≥ 0 and 0 otherwise, then z = 0 gives 1. If it says output 1 for z > 0 and 0 otherwise, then z = 0 gives 0. That single equality sign can change an entire training table.

The weighted sum is a real number. The threshold turns that number into a hard decision. Tap the diagram to enlarge.

Question. Implement binary AND with w₁ = w₂ = 0.6, b = −0.8 and output 1 when z ≥ 0.

Given

w₁ = w₂ = 0.6; bias b = −0.8.

Binary inputs; output 1 when z ≥ 0.

To find

Calculate the output for 00,01,10,11 and show that the result is AND.

Formula — which one and why

Threshold logic. Compute the weighted sum for every binary input combination and apply the threshold rule exactly. Hidden neurons can provide separate threshold decisions before the output combines them.

step(z) returns 1 at or above zero unless the question specifies a strict inequality; otherwise it returns 0.

$z=\sum_iw_ix_i+b$ LaTeX: z=\sum_iw_ix_i+b

$\operatorname{step}(z)=\begin{cases}1&z\ge0\\0&z<0\end{cases}$ LaTeX: \operatorname{step}(z)=\begin{cases}1&z\ge0\\0&z<0\end{cases}

Yaad rakhne ka cue: Har input row ka sum nikalo, phir threshold lagao.

Solution — step by step

Input (0, 0)

Step 1: multiply input 1 by its weight.

$0.6(0)=0$ LaTeX: 0.6(0)=0

Step 2: multiply input 2 by its weight.

$0.6(0)=0$ LaTeX: 0.6(0)=0

Step 3: add the two products.

$s=0+0=0$ LaTeX: s=0+0=0

Step 4: add the bias once.

$z=0+(-0.8)=-0.8$ LaTeX: z=0+(-0.8)=-0.8

Step 5: apply the stated threshold.

z = -0.8; output = 0 because output is 1 exactly when z ≥ 0.

Input (0, 1)

Step 1: multiply input 1 by its weight.

$0.6(0)=0$ LaTeX: 0.6(0)=0

Step 2: multiply input 2 by its weight.

$0.6(1)=0.6$ LaTeX: 0.6(1)=0.6

Step 3: add the two products.

$s=0+0.6=0.6$ LaTeX: s=0+0.6=0.6

Step 4: add the bias once.

$z=0.6+(-0.8)=-0.2$ LaTeX: z=0.6+(-0.8)=-0.2

Step 5: apply the stated threshold.

z = -0.2; output = 0 because output is 1 exactly when z ≥ 0.

Input (1, 0)

Step 1: multiply input 1 by its weight.

$0.6(1)=0.6$ LaTeX: 0.6(1)=0.6

Step 2: multiply input 2 by its weight.

$0.6(0)=0$ LaTeX: 0.6(0)=0

Step 3: add the two products.

$s=0.6+0=0.6$ LaTeX: s=0.6+0=0.6

Step 4: add the bias once.

$z=0.6+(-0.8)=-0.2$ LaTeX: z=0.6+(-0.8)=-0.2

Step 5: apply the stated threshold.

z = -0.2; output = 0 because output is 1 exactly when z ≥ 0.

Input (1, 1)

Step 1: multiply input 1 by its weight.

$0.6(1)=0.6$ LaTeX: 0.6(1)=0.6

Step 2: multiply input 2 by its weight.

$0.6(1)=0.6$ LaTeX: 0.6(1)=0.6

Step 3: add the two products.

$s=0.6+0.6=1.2$ LaTeX: s=0.6+0.6=1.2

Step 4: add the bias once.

$z=1.2+(-0.8)=0.4$ LaTeX: z=1.2+(-0.8)=0.4

Step 5: apply the stated threshold.

z = 0.4; output = 1 because output is 1 exactly when z ≥ 0.

Final answer: outputs are [0, 0, 0, 1] for inputs 00, 01, 10, 11: exactly AND.

Summary of the four cases

Inputs	Full net calculation	Net z	Output	AND target
0, 0	0.6(0) + 0.6(0) − 0.8	−0.8	0	0
0, 1	0.6(0) + 0.6(1) − 0.8	−0.2	0	0
1, 0	0.6(1) + 0.6(0) − 0.8	−0.2	0	0
1, 1	0.6(1) + 0.6(1) − 0.8	0.4	1	1

The neuron fires only when both inputs are 1. No weight update was requested, so this is a forward evaluation, not a training problem.

For binary targets, a common learning rule is w ← w + η(t − a)x and b ← b + η(t − a). For bipolar targets −1 and +1, the class notes use a different rule: on a mismatch, w ← w + η tx and b ← b + ηt; otherwise keep them. The notes give output 0 at zero threshold, +1 above it and −1 below it. Never combine the activation from one convention with an unexplained update from another.

Training table routine: copy current weights → calculate z → apply the stated threshold → compare output with target → compute every weight change → update bias → carry new values to the next row. Reset nothing between rows or epochs unless instructed.

Binary perceptron: target 1

Question. A binary perceptron has x = [1,2], weights [-0.2, 0.1], bias 0 and target 1. Use output 1 for net ≥ 0 and learning rate 0.1. Perform one update.

Given

x = [1,2]; w = [-0.2, 0.1]; b = 0; target = 1; η = 0.1.

To find

Net, threshold output, correction and new weights/bias.

Formula — which one and why

This binary learning rule uses target minus output. Carry the same correction into every input weight and the bias.

t is target; o is threshold output; Δ means the amount added to the current parameter.

$z=\sum_iw_ix_i+b$ LaTeX: z=\sum_iw_ix_i+b

$\Delta w_i=\eta(t-o)x_i$ LaTeX: \Delta w_i=\eta(t-o)x_i

$\Delta b=\eta(t-o)$ LaTeX: \Delta b=\eta(t-o)

Yaad rakhne ka cue: Target minus output; correct prediction ho toh correction zero.

Solution — step by step

Step 1: calculate the net input.

Step 2: apply the threshold.

Net = 0; output = 1 under the stated ≥ 0 rule.

Step 3: calculate the correction.

$t-o=1-1=0$ LaTeX: t-o=1-1=0

Step 4: update weight 1.

Step 5: update weight 2.

Step 6: update the bias.

$b_{\mathrm{new}}=0+0.1(0)=0$ LaTeX: b_{\mathrm{new}}=0+0.1(0)=0

Final answer: New weights = [-0.2,0.1]; new bias = 0.

Binary perceptron: target 0

Question. A binary perceptron has x = [1,2], weights [0.2, 0.1], bias 0 and target 0. Use output 1 for net ≥ 0 and learning rate 0.1. Perform one update.