Deep Learning · TEE · section

R26 Q6b

Re examination February 2026 fully solved

Question

Question (10 marks). For a 10-layer network using sigmoid: compute f(6), compute its derivative at 6, explain its effect on backpropagation, and calculate softmax for [2.3,4.5,3.2].

Answer — step by step

Given

Sigmoid is evaluated at z = 6 in a ten-layer network.

Softmax scores = [2.3,4.5,3.2].

To find

Calculate sigmoid and its derivative at 6, explain the gradient effect and calculate all three softmax probabilities.

Formula — which one and why

Sigmoid and its slope. Use sigmoid when the neuron is specified to squash its preactivation into (0,1). Once its output is known, its derivative needs only output × (1−output).

z = preactivation; p = sigmoid output; e is the exponential base, not a prediction error in this formula.

$p=\frac{1}{1+e^{-z}}$ LaTeX: p=\frac{1}{1+e^{-z}}

$\frac{dp}{dz}=p(1-p)$ LaTeX: \frac{dp}{dz}=p(1-p)

Yaad rakhne ka cue: Sigmoid output p; uska slope p × (1−p).

Stable softmax. Exponentiate each logit and divide by the sum. Subtracting the largest logit first keeps the exponentials numerically manageable and leaves the probabilities unchanged.

zⱼ = class j score; M = largest score; pⱼ = class j probability. All probabilities sum to 1.

$M=\max_j z_j$ LaTeX: M=\max_j z_j

$p_j=\frac{e^{z_j-M}}{\sum_k e^{z_k-M}}$ LaTeX: p_j=\frac{e^{z_j-M}}{\sum_k e^{z_k-M}}

Yaad rakhne ka cue: Largest score ghatao, exp nikalo, total se sabko divide karo.

Solution — step by step

f(6) = $\frac{1}{1+e^{-6}}$ = 0.997527377. Derivative = f(6)[1−f(6)] = 0.997527377 × 0.002472623 = 0.002466509. This is a saturated sigmoid with a very small slope. Repeatedly multiplying small activation derivatives and weight factors can weaken earlier-layer gradients; the exact gradient also depends on weights and all preactivations, so it is not simply this derivative raised to 10 unless those conditions are assumed.

Step 1 stabilize: subtract the largest logit 4.5 from every value: [-2.2, 0, -1.3]. This does not change softmax.

Step 2 exponentiate: exp values = [0.110803158, 1, 0.272531793].

Step 3 denominator: sum = 1.383334951.

Step 4 divide each value by this same sum: probabilities = [0.080098575, 0.722890721, 0.197010704]. Check: all are positive and their sum is 1 (apart from rounding); the largest logit gets the largest probability.