Deep Learning · TEE · section
Why XOR needs more than one perceptron
Neural-network foundations
XOR outputs 1 for [0,1] and [1,0], but 0 for [0,0] and [1,1]. On a graph the positive points occupy opposite corners. A single straight line cannot put both positive points on one side and both negative points on the other. A single perceptron has only one linear decision boundary.
A two-layer construction solves it. Let h₁ = step(x₁ + x₂ − 0.5), which is OR. Let h₂ = step(x₁ + x₂ − 1.5), which is AND. With step(z) = 1 for z ≥ 0, output XOR = step(h₁ − 2h₂ − 0.5). Output XNOR = step(−h₁ + 2h₂ + 0.5).
| x₁ x₂ | h₁ OR | h₂ AND | XOR net | XOR | XNOR |
|---|---|---|---|---|---|
| 0 0 | 0 | 0 | −0.5 | 0 | 1 |
| 0 1 | 1 | 0 | 0.5 | 1 | 0 |
| 1 0 | 1 | 0 | 0.5 | 1 | 0 |
| 1 1 | 1 | 1 | −1.5 | 0 | 1 |
More layers help only if appropriate nonlinearities are used. Two purely affine layers collapse into one affine map: W₂(W₁x + b₁) + b₂ = (W₂W₁)x + W₂b₁ + b₂. Affine means linear plus a bias.