# Counting parameters in a dense network

Subject: Deep Learning

Exam: MT2

Reader: https://deep-learning-study-room.vercel.app/#mt2/read/4/mt2-c4-s14

Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c4-s14/

**Definition.** A dense layer connects every input component to every receiving neuron. Its trainable parameters are these connection weights and the receiving neurons' independently trainable biases.

### Deriving the parameter count

For m inputs and n receiving neurons, each receiver needs m weights. There are therefore mn weights. With one bias per receiver, there are n additional parameters:



$$
P_{\mathrm{dense}}=mn+n=(m+1)n
$$



Activation values are computed from data and parameters; they are not additional trainable parameters. The input layer supplies data and does not acquire an extra bias merely because it appears in a diagram.

![Count all input-to-neuron connections and then one bias per receiving neuron.](https://deep-learning-study-room.vercel.app/assets/figures/v_dense_count.svg)

Diagram: Count all input-to-neuron connections and then one bias per receiving neuron.. Labels: Count connections separately from biases | Count connections separately from biases | A dense layer with 3 inputs and 2 receiving neurons. | x1 | x2 | x3 | a1 | + b1 | a2 | + b2 | 3 × 2 = 6 weights | Two receiving neurons also have 2 biases. Total = 6 + 2 = 8 parameters.

Count all input-to-neuron connections and then one bias per receiving neuron. Tap the diagram to enlarge.

### Complete numerical

**Question.** A network has 10 input features, three hidden layers with 5 neurons each, and 4 outputs. Every receiving neuron has a bias. Count its trainable parameters.

### Given

Layer sizes: 10 inputs → 5 hidden → 5 hidden → 5 hidden → 4 outputs.

Every hidden/output neuron has one bias.

### To find

Count weights and biases at each receiving layer, then the total.

### Formula — which one and why

**Dense-layer parameters.** Every output neuron receives every input, so multiply inputs by outputs for weights. Add one bias per receiving neuron. Sum separate layers.

n\_in = incoming features; n\_out = receiving neurons. The input layer itself has no learned receiving weights or biases.



$$
P=(n_{\mathrm{in}}+1)n_{\mathrm{out}}
$$



**Yaad rakhne ka cue:** Har output ke liye saare inputs plus ek bias.

### Solution — step by step

**Step 1: identify the consecutive layer widths.**



$$
10\longrightarrow5\longrightarrow5\longrightarrow5\longrightarrow4
$$



There are four trainable dense transformations. A connection between two hidden layers is counted once, as part of the receiving layer's parameters.

**Step 2: count weights and biases for each transformation.**

| Transformation | Weights | Biases | Layer total |
| --- | --- | --- | --- |
| Input → hidden 1 | 10 × 5 = 50 | 5 | 55 |
| Hidden 1 → hidden 2 | 5 × 5 = 25 | 5 | 30 |
| Hidden 2 → hidden 3 | 5 × 5 = 25 | 5 | 30 |
| Hidden 3 → output | 5 × 4 = 20 | 4 | 24 |

**Step 3: add the layer totals.**



$$
P=55+30+30+24=139
$$



Equivalently, the network has 120 weights and 19 biases, so 120 + 19 = 139.

**Final answer: 139 trainable parameters.** If a question explicitly removes biases, count only the 120 weights. Optimizer states such as Adam's moments are extra training storage, not extra parameters in this predictive network.