Deep Learning · TEE · section

R25 Q3b · Four CNN design explanations

February 2025 re-exam · complete solved paper

Question

Question. Explain(i)parameter sharing and sparse connections,(ii)5×5versus3×3kernel trade-offs,(iii)ReLU’s effect on learning,(iv)whether CNNs can work without pooling. (10 marks)

Answer — step by step

Part	Explanation
i	Local receptive fields create sparse connectivity. The same filter weights are reused across spatial positions, reducing parameters and recognizing a feature in different positions. This gives translation equivariance under appropriate boundary/stride conditions, not automatic complete invariance.
ii	At fixed channel counts,5×5uses25weights per channel-pair,3×3uses9. Two stride1 3×3layers yield an effective5×5receptive field with an extra nonlinearity and18C²weights when all channel widths equalC, versus25C². Intermediate channels, biases and compute/memory can change the comparison.
iii	ReLU passes positive values/gradient1 and zeros negative values/gradient0. It introduces nonlinearity and avoids positive-side sigmoid saturation, but can produce inactive/dead units. It does not eliminate all gradient problems.
iv	Yes. Strided convolutions can downsample; global pooling or other aggregation may form the head. Removing pooling without another reduction can increase feature-map sizes/compute. Pooling is a design choice, not a mathematical necessity.