# R26 Q7a

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/12/tee-c12-s14

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c12-s14/

## Question

**Question (10 marks).** Compare single-layer and multilayer feedforward networks and explain hidden nonlinearity. Compute the output for Input x = [0.5, 0.2, 0.1]. Each row below lists the incoming weights of one receiving neuron. W₁ is shown below; b₁ = [0.1, 0.2]; W₂ is shown below; b₂ = [0.5]. Hidden and output activations are sigmoid.

![Given matrices for this question](https://deep-learning-study-room.vercel.app/assets/figures/given_2fad289da9.svg)

Diagram: Given matrices for this question. Labels: GIVEN VALUES — read each matrix row from left to right | W₁ | 0.1 | 0.4 | 0.2 | 0.3 | 0.5 | 0.7 | W₂ | 0.6 | 0.9

Given matrices for this question Tap the diagram to enlarge.

## Answer — step by step

### Given

Input x = [0.5, 0.2, 0.1].

W₁ is printed with the question.

b₁ = [0.1, 0.2].

W₂ is printed with the question.

b₂ = [0.5].

Hidden and output activations are sigmoid.

### To find

Find all preactivations, hidden activations and final outputs. Find the sigmoid output and, if requested, its derivative.

### Formula — which one and why

**Forward propagation.** Evaluate one layer completely before supplying its activated outputs to the next layer. Each layer has its own weights and biases.

x = input vector; h = hidden outputs; W₁,b₁ = hidden parameters; W₂,b₂ = output parameters; f₁,f₂ = the stated activations.



$$
z_h=W_1x+b_1
$$





$$
h=f_1(z_h)
$$





$$
z_o=W_2h+b_2
$$





$$
o=f_2(z_o)
$$



**Yaad rakhne ka cue:** Pehli layer ka activated output hi agli layer ka input hai.

**Sigmoid and its slope.** Use sigmoid when the neuron is specified to squash its preactivation into (0,1). Once its output is known, its derivative needs only output × (1−output).

z = preactivation; p = sigmoid output; e is the exponential base, not a prediction error in this formula.



$$
p=\frac{1}{1+e^{-z}}
$$





$$
\frac{dp}{dz}=p(1-p)
$$



**Yaad rakhne ka cue:** Sigmoid output p; uska slope p × (1−p).

### Solution — step by step

A single perceptron forms a weighted sum and applies a threshold. Its decision boundary is linear: w·x+b = 0. It can learn linearly separable patterns such as AND and OR under suitable learning conditions, but cannot represent XOR with one linear separator.

![Opposite corners for XOR cannot be split by one straight line](https://deep-learning-study-room.vercel.app/assets/figures/xor.svg)

Diagram: Opposite corners for XOR cannot be split by one straight line. Labels: x1 | x2 | 0 | 1 | 1 | 0 | XOR labels occupy opposite corners | 0 | 1 | 0 | 1 | One straight decision boundary cannot separate the two classes.

Opposite corners for XOR cannot be split by one straight line Tap the diagram to enlarge.

For binary AND, w₁ = w₂ = 1 and b = −1.5 give preactivations −1.5, −0.5, −0.5 and 0.5 for 00, 10, 01 and 11. A step that returns 1 at nonnegative inputs produces 0,0,0,1.

For XOR, use h₁ = step(x₁+x₂−0.5) for OR and h₂ = step(x₁+x₂−1.5) for AND. Then XOR = step(h₁−2h₂−0.5). For inputs 00,01,10,11, the hidden pairs are (0,0), (1,0), (1,0), (1,1), producing 0,1,1,0. XNOR = step(−h₁+2h₂+0.5), producing 1,0,0,1. This is a multilayer construction, not a claim that a single perceptron solves XOR.

A multilayer feedforward network adds hidden nonlinear features before the final boundary. Without hidden nonlinearities, adding linear layers still produces one affine map.

**Step 1 map the arrows.** Each row of W₁ receives all inputs; each row of W₂ receives all hidden outputs. Bias is added once per receiving neuron.

**Hidden neuron 1:**



$$
z_{h_1} = (0.1 \times  0.5) + (0.4 \times  0.2) + (0.2 \times  0.1) + (0.1) = 0.25
$$





$$
h_{1}=\frac{1}{1+e^{-(0.25)}}=0.562176501
$$



**Hidden neuron 2:**



$$
z_{h_2} = (0.3 \times  0.5) + (0.5 \times  0.2) + (0.7 \times  0.1) + (0.2) = 0.52
$$





$$
h_{2}=\frac{1}{1+e^{-(0.52)}}=0.627147766
$$



**Output neuron 1:**



$$
z_{o_1} = (0.6 \times  0.562176501) + (0.9 \times  0.627147766) + (0.5) = 1.40173889
$$





$$
o_{1}=\frac{1}{1+e^{-(1.40173889)}}=0.802459679
$$