R23 Q6b

Answer — step by step

Given

Input = 28 × 28 × 192; 32 same-padded 5 × 5 filters.

A preceding 1 × 1 convolution would reduce to C channels.

C is not supplied, so first give the symbolic bottleneck count; 16 channels is only an illustrative assumption.

To find

Calculate direct cost and express bottleneck cost in C; explain when the bottleneck saves computation.

Formula — which one and why

Multiply–accumulate count. Each output position and output channel uses K²C_in products. A bottleneck requires two separate counts, one for the 1×1 reduction and one for the following spatial convolution.

H_out,W_out = output map size; C_in,C_out = incoming/outgoing channels; K = kernel width. Count one multiply–accumulate as one MAC; bias additions are excluded.

LaTeX: \mathrm{MACs}=H_{\mathrm{out}}W_{\mathrm{out}}C_{\mathrm{out}}K^2C_{\mathrm{in}}

LaTeX: \mathrm{saving\ percentage}=100\frac{\mathrm{direct}-\mathrm{reduced}}{\mathrm{direct}}

Yaad rakhne ka cue: Har output cell ka kaam × kitne output cells.

Solution — step by step

Direct path

Step 2: direct path.

output stays 28 × 28 with 32 channels.

Step 3: direct path.

MACs = 28 × 28 × 192 × 32 × 5 × 5 = 120,422,400.

Bottleneck path with 16 reduction channels

Step 4: bottleneck path with 16 reduction channels.

1 × 1 stage costs 28 × 28 × 192 × 16 = 2,408,448 MACs.

Step 5: bottleneck path with 16 reduction channels.

The 5 × 5 stage costs 28 × 28 × 16 × 32 × 25 = 10,035,200 MACs.

Step 6: bottleneck path with 16 reduction channels.

Total = 12,443,648.

Saving

Step 7: saving.

120,422,400−12,443,648 = 107,978,752 MACs, or 107,978,752/120,422,400 × 100 = 89.666667%. This counts multiplications with accumulation as MACs and omits biases/nonlinearities. Padding preserves spatial size. If the reduction width is not specified, call it r: total = 784 × 192r + 784 × 25 × 32r = 777,728r

Step 8: saving.

there is no unique numeric bottleneck cost until r is chosen.

The 16-channel bottleneck is an explicitly chosen illustrative value, consistent with a related paper; the symbolic 777,728 r answer is required when r has not been specified.

Step 9: calculate this quantity.

An Inception module captures features at multiple scales in parallel. A typical module has a 1 × 1 branch, a 1 × 1 reduction followed by 3 × 3 convolution, a 1 × 1 reduction followed by 5 × 5 convolution, and a pooling branch followed by 1 × 1 projection. Padding and strides preserve compatible spatial sizes so branch outputs can be concatenated along the channel axis.

Diagram: Parallel branches preserve height and width then concatenate channels. Labels: Input | 1 × 1 | 1 × 1 reduce | 3 × 3 | 1 × 1 reduce | 5 × 5 | 3 × 3 pool | 1 × 1 | Concatenate channels | All branch height/width values must match.

Step 10: calculate this quantity.

A 1 × 1 convolution mixes input channels at each position. With fewer output channels, it forms a bottleneck before an expensive larger kernel. ReLU following it adds nonlinearity. This reduces the number of input channels processed by 3 × 3 or 5 × 5 kernels while preserving useful learned combinations.

For H × W spatial size, Cin input channels and Cout output filters, a K × K convolution uses H × W × Cin × Cout × K² multiply-accumulates when the output spatial size stays H × W. A bottleneck changes the calculation to H × W × Cin × r + H × W × r × Cout × K². A multiply-accumulate is counted as one MAC here; some FLOP conventions count it as two operations. The benefit depends on the selected bottleneck width r. Compared with AlexNet's sequential kernels, Inception explicitly combines local and larger-scale features within one module.