# R23 Q6b

Subject: Deep Learning

Exam: MT2

Reader: https://deep-learning-study-room.vercel.app/#mt2/read/16/mt2-c16-s8

Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c16-s8/

## Question

**Question (10 marks).** A 28 × 28 × 192 input is convolved with 32 same-padded 5 × 5 filters to produce 28 × 28 × 32. Count computations. What happens if a 1 × 1 convolution precedes it? The reduction-channel count is not supplied.

## Answer — step by step

### Given

Input = 28 × 28 × 192; 32 same-padded 5 × 5 filters.

A preceding 1 × 1 convolution would reduce to C channels.

C is not supplied, so first give the symbolic bottleneck count; 16 channels is only an illustrative assumption.

### To find

Calculate direct cost and express bottleneck cost in C; explain when the bottleneck saves computation.

### Formula — which one and why

**Multiply–accumulate count.** Each output position and output channel uses K²C\_in products. A bottleneck requires two separate counts, one for the 1×1 reduction and one for the following spatial convolution.

H\_out,W\_out = output map size; C\_in,C\_out = incoming/outgoing channels; K = kernel width. Count one multiply–accumulate as one MAC; bias additions are excluded.



$$
\mathrm{MACs}=H_{\mathrm{out}}W_{\mathrm{out}}C_{\mathrm{out}}K^2C_{\mathrm{in}}
$$





$$
\mathrm{saving\ percentage}=100\frac{\mathrm{direct}-\mathrm{reduced}}{\mathrm{direct}}
$$



**Yaad rakhne ka cue:** Har output cell ka kaam × kitne output cells.

### Solution — step by step

### Direct path

**Step 2: direct path.**

output stays 28 × 28 with 32 channels.

**Step 3: direct path.**

MACs = 28 × 28 × 192 × 32 × 5 × 5 = **120,422,400**.

### Bottleneck path with 16 reduction channels

**Step 4: bottleneck path with 16 reduction channels.**

1 × 1 stage costs 28 × 28 × 192 × 16 = 2,408,448 MACs.

**Step 5: bottleneck path with 16 reduction channels.**

The 5 × 5 stage costs 28 × 28 × 16 × 32 × 25 = 10,035,200 MACs.

**Step 6: bottleneck path with 16 reduction channels.**

Total = **12,443,648**.

### Saving

**Step 7: saving.**

120,422,400−12,443,648 = 107,978,752 MACs, or 107,978,752/120,422,400 × 100 = **89.666667%**. This counts multiplications with accumulation as MACs and omits biases/nonlinearities. Padding preserves spatial size. If the reduction width is not specified, call it r: total = 784 × 192r + 784 × 25 × 32r = **777,728r**

**Step 8: saving.**

there is no unique numeric bottleneck cost until r is chosen.

The 16-channel bottleneck is an explicitly chosen illustrative value, consistent with a related paper; the symbolic 777,728 r answer is required when r has not been specified.

**Step 9: calculate this quantity.**

An Inception module captures features at multiple scales in parallel. A typical module has a 1 × 1 branch, a 1 × 1 reduction followed by 3 × 3 convolution, a 1 × 1 reduction followed by 5 × 5 convolution, and a pooling branch followed by 1 × 1 projection. Padding and strides preserve compatible spatial sizes so branch outputs can be concatenated along the channel axis.

![Parallel branches preserve height and width then concatenate channels](https://deep-learning-study-room.vercel.app/assets/figures/inception.svg)

Diagram: Parallel branches preserve height and width then concatenate channels. Labels: Input | 1 × 1 | 1 × 1 reduce | 3 × 3 | 1 × 1 reduce | 5 × 5 | 3 × 3 pool | 1 × 1 | Concatenate channels | All branch height/width values must match.

Parallel branches preserve height and width then concatenate channels Tap the diagram to enlarge.

**Step 10: calculate this quantity.**

A 1 × 1 convolution mixes input channels at each position. With fewer output channels, it forms a bottleneck before an expensive larger kernel. ReLU following it adds nonlinearity. This reduces the number of input channels processed by 3 × 3 or 5 × 5 kernels while preserving useful learned combinations.

For H × W spatial size, Cin input channels and Cout output filters, a K × K convolution uses H × W × Cin × Cout × K² multiply-accumulates when the output spatial size stays H × W. A bottleneck changes the calculation to H × W × Cin × r + H × W × r × Cout × K². A multiply-accumulate is counted as one MAC here; some FLOP conventions count it as two operations. The benefit depends on the selected bottleneck width r. Compared with AlexNet's sequential kernels, Inception explicitly combines local and larger-scale features within one module.