# What one artificial neuron does

Subject: Deep Learning

Exam: MT2

Reader: https://deep-learning-study-room.vercel.app/#mt2/read/4/mt2-c4-s5

Canonical: https://deep-learning-study-room.vercel.app/read/dl/mt2/section/mt2-c4-s5/

**Definition.** An artificial neuron is a calculation unit that multiplies each input by a matching weight, adds the products and a bias, and applies an activation function to produce its output.

### What goes into a neuron?

An input is a number supplied to the calculation. With three inputs, x = [2,1,3] simply lists those three numbers in order. The matching weight list tells the neuron which multiplier to use for each input. A negative weight contributes a negative amount when its input is positive.

The **bias** is one extra adjustable number added to the total. The total before activation is called z or **preactivation**. The **activation function** is the final rule applied to z: for example, ReLU replaces a negative z with zero, while sigmoid turns z into a number between 0 and 1. The resulting output is called a or the **activation**.

**Yaad rakhne ka cue:** inputs × apne weights → products ka sum → ek bias → activation → output.

### Mathematical model and notation



$$
z=\sum_{i=1}^{d}w_ix_i+b=w^Tx+b
$$





$$
a=f(z)
$$



| Symbol | Meaning and role |
| --- | --- |
| xᵢ | Input component i; the neuron receives d such components |
| wᵢ | Trainable weight multiplying that input component |
| b | Trainable bias added once to this neuron's total |
| z | Preactivation: the result before applying the activation function |
| f | Chosen activation function, such as sigmoid or ReLU |
| a | Activation: the neuron's output after applying f |

The dot product wᵀx is a compact notation for the sum of matching weight–input products. A positive weight and a negative weight can make opposite contributions for the same positive input. A larger weight magnitude means a larger contribution for a fixed input; it is not, by itself, a general measure of feature importance across differently scaled inputs.

### Why both a bias and an activation are needed

The bias changes the preactivation independently of x. In a one-input affine function z = wx + b, it is the intercept. In a classifier with a decision boundary z = 0, a bias allows that boundary to be shifted instead of forcing it through the origin.

The activation determines how z is transformed. Sigmoid produces a value between 0 and 1; ReLU retains positive preactivations and sets negative ones to zero. Nonlinear activations let stacked layers represent nonlinear functions. The preactivation and activation are distinct quantities even when a particular ReLU calculation gives them equal values.

![Read the neuron in order: input contributions → affine sum z → activation function f → output a. Add one bias at the receiving neuron.](https://deep-learning-study-room.vercel.app/assets/figures/neuron.svg)

Diagram: Read the neuron in order: input contributions → affine sum z → activation function f → output a. Add one bias at the receiving neuron.. Labels: INPUTS | WEIGHTED SUM | ACTIVATION | x1 | w1 | x2 | w2 | x3 | w3 | z = Σwx + b | a = f(z) | Add one bias b | Multiply → add → activate

Read the neuron in order: input contributions → affine sum z → activation function f → output a. Add one bias at the receiving neuron. Tap the diagram to enlarge.

### Worked numerical

**Question.** A neuron has x = [2, 1, 3]ᵀ, w = [0.4, −0.2, 0.5]ᵀ and b = 0.1. Calculate its preactivation and sigmoid output.

### Given

Inputs = [2,1,3]; matching weights = [0.4,−0.2,0.5].

Bias = 0.1; activation = sigmoid.

### To find

Find the preactivation z, then the sigmoid output.

### Formula — which one and why

**Weighted sum and activation.** Each input is multiplied by its connection weight. Add these contributions, add the bias once, then apply the stated activation.

xᵢ = input i; wᵢ = its weight; b = bias; z = sum before activation; a = output after activation; f = the specified activation function.



$$
z=\sum_iw_ix_i+b
$$





$$
a=f(z)
$$



**Yaad rakhne ka cue:** Input × weight, sabka sum, bias once, phir activation.

**Sigmoid output.** The question specifies sigmoid, so turn the complete weighted sum into an output between 0 and 1. Negate z, evaluate the exponential, add 1, then take the reciprocal.

z = sum before activation; a = sigmoid output; e ≈ 2.71828 is the exponential base.



$$
a=\frac1{1+e^{-z}}
$$



**Yaad rakhne ka cue:** Minus-z ki e power, plus one, phir one divided by whole denominator.

### Solution — step by step

**Step 1: multiply input 1 by weight 1.**



$$
x_1w_1=2(0.4)=0.8
$$



**Step 2: multiply input 2 by weight 2.**



$$
x_2w_2=1(-0.2)=-0.2
$$



The minus sign belongs to the weight, so the contribution stays negative.

**Step 3: multiply input 3 by weight 3.**



$$
x_3w_3=3(0.5)=1.5
$$



**Step 4: add the three contributions.**



$$
s=0.8+(-0.2)+1.5=0.6+1.5=2.1
$$



**Step 5: add the bias once.**



$$
z=s+b=2.1+0.1=2.2
$$



This is the preactivation. It is not yet the requested sigmoid output.

**Step 6: negate z for the exponential.**



$$
-z=-2.2
$$





$$
e^{-z}=e^{-2.2}\approx0.110803158
$$



**Step 7: calculate the complete denominator.**



$$
1+e^{-z}=1+0.110803158=1.110803158
$$



**Step 8: divide one by that denominator.**



$$
a=\frac1{1+e^{-z}}=\frac1{1.110803158}\approx0.900249511
$$



**Final answer:** sigmoid output ≈ 0.900249511. It lies between 0 and 1. The raw sum was 2.2, whereas the activated output is approximately 0.90025.

### From one neuron to a layer

With m receiving neurons and d input components, W has shape m × d. Row j contains the incoming weights of neuron j. The bias vector has m entries, one for each receiving neuron:



$$
z=Wx+b,\qquad a=f(z)
$$



Sigmoid, tanh and ReLU apply independently to each component. Softmax is different: it transforms a whole vector of output logits together.

### Neuron example 2: ReLU output

**Question. A neuron receives inputs [2], weights [3] and bias 1. Calculate its preactivation and ReLU output.**

### Given

Inputs = [2].

Weights = [3]; bias = 1.

Activation = ReLU.

### To find

z before activation and a after activation.

### Formula — which one and why

The weighted sum combines the inputs. The output must then use the activation explicitly requested.

z names the sum; a names the activated result. They are not always equal.



$$
z=\sum_i w_ix_i+b
$$





$$
a=\max(0,z)
$$



**Yaad rakhne ka cue:** Sum alag, activation alag; activation lagana mat bhool.

### Solution — step by step

**Step 1: multiply input 1 by its own weight.**



$$
(2)(3)=6
$$



**Step 2: add the input contributions.**



$$
s=(6)=6
$$



**Step 3: add the bias once.**



$$
z=6+(1)=7
$$



**Step 4: apply the stated activation.**



$$
a=\max(0,7)=7
$$



**Final answer:** z = 7; a = 7.

### Neuron example 3: ReLU output

**Question. A neuron receives inputs [2, 1], weights [-1, 0.5] and bias 0.2. Calculate its preactivation and ReLU output.**

### Given

Inputs = [2, 1].

Weights = [-1, 0.5]; bias = 0.2.

Activation = ReLU.

### To find

z before activation and a after activation.

### Formula — which one and why

The weighted sum combines the inputs. The output must then use the activation explicitly requested.

z names the sum; a names the activated result. They are not always equal.



$$
z=\sum_i w_ix_i+b
$$





$$
a=\max(0,z)
$$



**Yaad rakhne ka cue:** Sum alag, activation alag; activation lagana mat bhool.

### Solution — step by step

**Step 1: multiply input 1 by its own weight.**



$$
(2)(-1)=-2
$$



**Step 2: multiply input 2 by its own weight.**



$$
(1)(0.5)=0.5
$$



**Step 3: add the input contributions.**



$$
s=(-2)+(0.5)=-1.5
$$



**Step 4: add the bias once.**



$$
z=-1.5+(0.2)=-1.3
$$



**Step 5: apply the stated activation.**



$$
a=\max(0,-1.3)=0
$$



**Final answer:** z = -1.3; a = 0.

### Neuron example 4: sigmoid output

**Question. A neuron receives inputs [1, 2], weights [0.2, -0.1] and bias 0. Calculate its preactivation and sigmoid output.**

### Given

Inputs = [1, 2].

Weights = [0.2, -0.1]; bias = 0.

Activation = sigmoid.

### To find

z before activation and a after activation.

### Formula — which one and why

The weighted sum combines the inputs. The output must then use the activation explicitly requested.

z names the sum; a names the activated result. They are not always equal.



$$
z=\sum_i w_ix_i+b
$$





$$
a=\frac1{1+e^{-z}}
$$



**Yaad rakhne ka cue:** Sum alag, activation alag; activation lagana mat bhool.

### Solution — step by step

**Step 1: multiply input 1 by its own weight.**



$$
(1)(0.2)=0.2
$$



**Step 2: multiply input 2 by its own weight.**



$$
(2)(-0.1)=-0.2
$$



**Step 3: add the input contributions.**



$$
s=(0.2)+(-0.2)=0
$$



**Step 4: add the bias once.**



$$
z=0+(0)=0
$$



**Step 5: apply the stated activation.**



$$
a=\frac1{1+e^{0}}=\frac1{1+1}=0.5
$$



**Final answer:** z = 0; a = 0.5.