# Parameters are different from activation values

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/7/tee-c7-s8

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c7-s8/

**Definition.** Parameters are learned coefficients shared across evaluations. Activations are input-dependent values computed during a forward pass. The number of activation values therefore differs from the number of trainable parameters.

### Explanation and working

An activation is a number produced for a particular input. A parameter is a learned number reused across inputs. A 100 × 100 feature map can be produced by a filter with only a few weights.

![The ten learned parameters are reused at nine spatial positions. The resulting nine numbers are activations.](https://deep-learning-study-room.vercel.app/assets/figures/v_parameters_activations.svg)

Diagram: The ten learned parameters are reused at nine spatial positions. The resulting nine numbers are activations.. Labels: Learned parameters are not output activations | Learned parameters are not output activations | Example: a 5 × 5 × 1 input with one 3 × 3 filter, stride 1, no padding. | Learned filter | 3 × 3 × 1 = 9 weights | Plus one bias | Total parameters = 10 | Output feature map | (5 − 3)/1 + 1 = 3 | Shape = 3 × 3 × 1 | Total activations = 9 | The same ten parameters are reused to calculate all nine output values.

The ten learned parameters are reused at nine spatial positions. The resulting nine numbers are activations. Tap the diagram to enlarge.



$$
P_{\mathrm{conv}}=(K_hK_wC_{\mathrm{in}}+1)C_{\mathrm{out}}
$$



The +1 is one bias per filter. If bias is explicitly disabled, remove it. Height and width of the image do not enter the parameter formula; they enter the number of output activations and the amount of computation.

### One filter, then all filters

**Question. A 3 × 3 convolution receives 3 input channels and has 16 filters, each with one bias. Find its trainable parameters and output-channel count.**

### Given

Kernel = 3 × 3; input channels = 3; filters = 16; one bias per filter.

### To find

Weights, biases, total trainable parameters and output channels.

### Formula — which one and why

Each filter spans every input channel and owns one bias. Spatial reuse does not create extra weights.

Kₕ,Kw are kernel dimensions; Cᵢₙ is input depth; Cₒᵤₜ is the number of filters.



$$
P=(K_hK_wC_{\mathrm{in}}+1)C_{\mathrm{out}}
$$



**Yaad rakhne ka cue:** Ek filter ka count banao, phir filters se multiply.

### Solution — step by step

**Step 1: count weights in one filter.**



$$
3(3)(3)=27
$$



**Step 2: include its one bias.**



$$
27+1=28
$$



**Step 3: count all weights.**



$$
27(16)=432
$$



**Step 4: count all biases.**



$$
1(16)=16
$$



**Step 5: add and state channel count.**



$$
P=432+16=448
$$



Each filter produces one channel, so output channels = 16.

**Final answer:** 432 weights + 16 biases = 448 trainable parameters; 16 output channels.



$$
P_{\mathrm{pool}}=0,\quad P_{\mathrm{flatten}}=0,\quad P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}
$$



Flatten rearranges H × W × C activation values into one vector of length HWC. It does not learn anything. A following dense layer connects to every one of those values.