Deep Learning · MT2 · section

CNN versus a fully connected image network

Convolutional neural networks

Definition. A convolutional neural network uses local receptive fields and spatial parameter sharing, unlike a dense layer's independent connection weights. These architectural constraints exploit spatial structure and can greatly reduce parameter counts.

Explanation and working

A fully connected layer can learn image features, but flattening pixels and connecting every pixel to every neuron ignores the useful local structure built into a CNN and can require many parameters. CNNs exploit local connections and shared filters. They learn hierarchical features and reuse a pattern detector at many locations.

Diagram: Compare where connections go and which weights are reused.. Labels: Fully connected and convolutional layers connect differently | Fully connected and convolutional layers connect differently | Both learn weights; the connection pattern changes. | Fully connected image layer | Every output sees every input. | Each connection has a weight. | Spatial layout is not built in. | Convolutional layer | A filter sees a local patch. | Reuse it at many positions. | Preserve a spatial feature map. | Parameter sharing makes a convolution economical when the same feature can occur | in many places.

For a 32 × 32 × 3 input, a dense layer with 100 neurons has (3,072 + 1) × 100 = 307,300 parameters. A convolution with 100 filters of size 3 × 3 has (27 + 1) × 100 = 2,800 parameters. The outputs are different shapes, so this is a parameter-efficiency illustration, not two identical computational tasks.