# F22 Q4a

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/15/tee-c15-s9

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c15-s9/

## Question

**Question (10 marks).** An Inception branch receives 28 × 28 × 192, uses 16 filters of 1 × 1 then 32 filters of 5 × 5 with spatial size preserved. Compute operations and explain 1 × 1 convolution.

## Answer — step by step

### Given

An Inception branch receives 28 × 28 × 192, uses 16 filters of 1 × 1 then 32 filters of 5 × 5 with spatial size preserved.

### To find

Find direct and bottleneck computation counts and the requested saving.

### Formula — which one and why

**Multiply–accumulate count.** Each output position and output channel uses K²C\_in products. A bottleneck requires two separate counts, one for the 1×1 reduction and one for the following spatial convolution.

H\_out,W\_out = output map size; C\_in,C\_out = incoming/outgoing channels; K = kernel width. Count one multiply–accumulate as one MAC; bias additions are excluded.



$$
\mathrm{MACs}=H_{\mathrm{out}}W_{\mathrm{out}}C_{\mathrm{out}}K^2C_{\mathrm{in}}
$$





$$
\mathrm{saving\ percentage}=100\frac{\mathrm{direct}-\mathrm{reduced}}{\mathrm{direct}}
$$



**Yaad rakhne ka cue:** Har output cell ka kaam × kitne output cells.

### Solution — step by step

### Direct path

**Step 2: direct path.**

output stays 28 × 28 with 32 channels.

**Step 3: direct path.**

MACs = 28 × 28 × 192 × 32 × 5 × 5 = **120,422,400**.

### Bottleneck path with 16 reduction channels

**Step 4: bottleneck path with 16 reduction channels.**

1 × 1 stage costs 28 × 28 × 192 × 16 = 2,408,448 MACs.

**Step 5: bottleneck path with 16 reduction channels.**

The 5 × 5 stage costs 28 × 28 × 16 × 32 × 25 = 10,035,200 MACs.

**Step 6: bottleneck path with 16 reduction channels.**

Total = **12,443,648**.

### Saving

**Step 7: saving.**

120,422,400−12,443,648 = 107,978,752 MACs, or 107,978,752/120,422,400 × 100 = **89.666667%**. This counts multiplications with accumulation as MACs and omits biases/nonlinearities. Padding preserves spatial size. If the reduction width is not specified, call it r: total = 784 × 192r + 784 × 25 × 32r = **777,728r**

**Step 8: saving.**

there is no unique numeric bottleneck cost until r is chosen.

**Step 9: calculate this quantity.**

An Inception module captures features at multiple scales in parallel. A typical module has a 1 × 1 branch, a 1 × 1 reduction followed by 3 × 3 convolution, a 1 × 1 reduction followed by 5 × 5 convolution, and a pooling branch followed by 1 × 1 projection. Padding and strides preserve compatible spatial sizes so branch outputs can be concatenated along the channel axis.

![Parallel branches preserve height and width then concatenate channels](https://deep-learning-study-room.vercel.app/assets/figures/inception.svg)

Diagram: Parallel branches preserve height and width then concatenate channels. Labels: Input | 1 × 1 | 1 × 1 reduce | 3 × 3 | 1 × 1 reduce | 5 × 5 | 3 × 3 pool | 1 × 1 | Concatenate channels | All branch height/width values must match.

Parallel branches preserve height and width then concatenate channels Tap the diagram to enlarge.

**Step 10: calculate this quantity.**

A 1 × 1 convolution mixes input channels at each position. With fewer output channels, it forms a bottleneck before an expensive larger kernel. ReLU following it adds nonlinearity. This reduces the number of input channels processed by 3 × 3 or 5 × 5 kernels while preserving useful learned combinations.

For H × W spatial size, Cin input channels and Cout output filters, a K × K convolution uses H × W × Cin × Cout × K² multiply-accumulates when the output spatial size stays H × W. A bottleneck changes the calculation to H × W × Cin × r + H × W × r × Cout × K². A multiply-accumulate is counted as one MAC here; some FLOP conventions count it as two operations. The benefit depends on the selected bottleneck width r. Compared with AlexNet's sequential kernels, Inception explicitly combines local and larger-scale features within one module.