# Drill 39 Bottleneck cost with new values

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s39

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s39/

## Question

**Question.** A 16 × 16 × 64 map feeds 32 same-padded 3 × 3 filters. Compare direct MACs with an 8-channel 1 × 1 bottleneck first.

## Answer — step by step

### Given

Input = 16 × 16 × 64.

Direct branch: 32 same-padded 3 × 3 filters.

Alternative: 1 × 1 reduction to 8 channels, then the same 32-filter 3 × 3 layer.

Spatial size remains 16 × 16; exclude bias additions.

### To find

Find both MAC counts, absolute saving and percentage saving.

### Formula — which one and why

**Multiply–accumulate count.** Each output position and output channel uses K²C\_in products. A bottleneck requires two separate counts, one for the 1×1 reduction and one for the following spatial convolution.

H\_out,W\_out = output map size; C\_in,C\_out = incoming/outgoing channels; K = kernel width. Count one multiply–accumulate as one MAC; bias additions are excluded.



$$
\mathrm{MACs}=H_{\mathrm{out}}W_{\mathrm{out}}C_{\mathrm{out}}K^2C_{\mathrm{in}}
$$





$$
\mathrm{saving\ percentage}=100\frac{\mathrm{direct}-\mathrm{reduced}}{\mathrm{direct}}
$$



**Yaad rakhne ka cue:** Har output cell ka kaam × kitne output cells.

### Solution — step by step

### Meaning and formula

A MAC is one multiply–accumulate operation. For the convention used here, count the weight–input products over every output location. Bias additions are omitted. A 1 × 1 bottleneck reduces channels before the expensive 3 × 3 layer.



$$
\mathrm{MACs}=H_{\mathrm{out}}W_{\mathrm{out}}C_{\mathrm{out}}K^2C_{\mathrm{in}}
$$



### Step 1: direct 3 × 3 convolution

Each output uses 3 × 3 × 64 = 576 weight–input pairs. There are 16 × 16 × 32 = 8,192 outputs.



$$
\mathrm{MACs}_{\mathrm{direct}}=8192\times576=4{,}718{,}592
$$



### Step 2: 1 × 1 reduction from 64 to 8 channels



$$
\mathrm{MACs}_{1\times1}=16\times16\times8\times1\times1\times64=131{,}072
$$



### Step 3: 3 × 3 convolution now receives only 8 channels



$$
\mathrm{MACs}_{3\times3}=16\times16\times32\times3\times3\times8=589{,}824
$$



### Step 4: add the two bottleneck stages



$$
\mathrm{MACs}_{\mathrm{bottleneck}}=131{,}072+589{,}824=720{,}896
$$



### Step 5: calculate the saving



$$
\mathrm{saving}=4{,}718{,}592-720{,}896=3{,}997{,}696
$$





$$
\mathrm{saving\ percent}=\frac{3{,}997{,}696}{4{,}718{,}592}\times100=84.722222\%
$$



**Final answer:** direct 4,718,592 MACs; bottleneck 720,896 MACs; approximately 84.72% fewer. The output shape stays 16 × 16 × 32.