# Inception concatenation and residual addition

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/8/tee-c8-s7

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c8-s7/

**Definition.** Channel concatenation joins feature tensors while preserving their separate channels. Residual addition sums corresponding entries. These operations impose different shape constraints and produce different output depths.

### Explanation and working

Concatenation joins feature channels; elementwise addition sums matching entries. They are different operations.

![Concatenation stacks channels. Residual addition combines matching values and preserves the matched shape.](https://deep-learning-study-room.vercel.app/assets/figures/v_join_add.svg)

Diagram: Concatenation stacks channels. Residual addition combines matching values and preserves the matched shape.. Labels: Concatenation and addition produce different shapes | Concatenation and addition produce different shapes | Use height and width 14 × 14 throughout. | CONCATENATE four branches | Channels: 16 + 32 + 8 + 8 | Result: 14 × 14 × 64 | Stack all 64 channels. | ADD two matching tensors | 14 × 14 × 64 + 14 × 14 × 64 | Result: 14 × 14 × 64 | Add each matching pair. | A 14 × 14 × 32 shortcut needs a projection before addition to a 14 × 14 × 64 | branch.

Concatenation stacks channels. Residual addition combines matching values and preserves the matched shape. Tap the diagram to enlarge.

Four Inception branches of spatial size 14 × 14 have 16, 32, 8 and 8 channels. First add the channel counts: 16 + 32 = 48; 48 + 8 = 56; 56 + 8 = 64. Height and width stay 14. The concatenated output is 14 × 14 × 64.

A residual addition requires matching shapes. A 14 × 14 × 64 tensor cannot be directly added to a 14 × 14 × 32 tensor. A projection, often a 1 × 1 convolution, can transform one path to the matching shape before addition.