Deep Learning · MT2 · section

Inception concatenation and residual addition

CNN architectures and transfer learning

Definition. Channel concatenation joins feature tensors while preserving their separate channels. Residual addition sums corresponding entries. These operations impose different shape constraints and produce different output depths.

Explanation and working

Concatenation joins feature channels; elementwise addition sums matching entries. They are different operations.

Diagram: Concatenation stacks channels. Residual addition combines matching values and preserves the matched shape.. Labels: Concatenation and addition produce different shapes | Concatenation and addition produce different shapes | Use height and width 14 × 14 throughout. | CONCATENATE four branches | Channels: 16 + 32 + 8 + 8 | Result: 14 × 14 × 64 | Stack all 64 channels. | ADD two matching tensors | 14 × 14 × 64 + 14 × 14 × 64 | Result: 14 × 14 × 64 | Add each matching pair. | A 14 × 14 × 32 shortcut needs a projection before addition to a 14 × 14 × 64 | branch.

Four Inception branches of spatial size 14 × 14 have 16, 32, 8 and 8 channels. First add the channel counts: 16 + 32 = 48; 48 + 8 = 56; 56 + 8 = 64. Height and width stay 14. The concatenated output is 14 × 14 × 64.

A residual addition requires matching shapes. A 14 × 14 × 64 tensor cannot be directly added to a 14 × 14 × 32 tensor. A projection, often a 1 × 1 convolution, can transform one path to the matching shape before addition.