# Residual-block order and architecture qualifications

Subject: Deep Learning

Exam: MT2

Reader: https://deep-learning-study-room.vercel.app/#mt2/read/8/mt2-c8-s16

Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c8-s16/

The original post-activation basic block has this sequence: **convolution → BN → ReLU → convolution → BN → add shortcut → ReLU**. Thus the second branch convolution is not followed by a separate ReLU before the addition in this particular block. A pre-activation ResNet is another design; use the stated diagram.

![Add the shortcut to the residual branch, then apply the post-addition ReLU when the specified block requires it.](https://deep-learning-study-room.vercel.app/assets/figures/residual.svg)

Diagram: Add the shortcut to the residual branch, then apply the post-addition ReLU when the specified block requires it.. Labels: x | Learned layers F(x) | + | y | Shortcut S(x): identity or projection | y = F(x) + S(x) | S(x) = x for identity; S(x) = W\_s x for projection. | Add matching shapes; do not concatenate.

Add the shortcut to the residual branch, then apply the post-addition ReLU when the specified block requires it. Tap the diagram to enlarge.

An identity shortcut has no learned parameters. A projection shortcut changes the shape using learned weights. Original ImageNet ResNets include an initial max-pooling operation; a basic residual block does not need a max-pooling layer inside it. ResNet-18/34 use two-convolution basic blocks, whereas deeper standard variants use bottleneck blocks. Avoid treating every ResNet as the same two-layer diagram.

The main advantage is easier optimization of deep networks: shortcuts provide an additional information and gradient path. This reduces the degradation problem but does not prove immunity to every vanishing/exploding-gradient case. Large networks still need compute, memory, suitable data and regularization. Reusing early features can help; it does not guarantee that every deeper model generalizes better.