# Drill 49 Mixed CNN dimensions softmax and loss

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s49

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s49/

## Question

**Question.** Input 8 × 8 × 3 → Conv 3, stride 1, padding 1, 2 filters → Pool 2, stride 2 → Flatten → Dense 3. Logits [0,1,−1], true class is the second class (one-hot target [0,1,0]). Find shapes, parameters, probabilities and CCE.

## Answer — step by step

### Given

Input = 8 × 8 × 3.

Conv: kernel 3, stride 1, padding 1, 2 filters.

Pool: kernel 2, stride 2; flatten; dense layer with 3 outputs.

Logits = [0,1,−1]; true class is the second class (one-hot target [0,1,0]).

### To find

Find shapes, parameter counts, probabilities and CCE.

### Formula — which one and why

**Complete CNN calculation.** Carry each output shape into the next layer. Convolution changes spatial size and channel count; pooling changes spatial size; flattening converts all activations into dense-layer inputs.

H,W,C = feature-map height, width and channels; K = kernel size; S = stride; P\_pad = padding. One bias is assumed for each learned filter or dense neuron unless specified otherwise.



$$
N_{\mathrm{out}}=\left\lfloor\frac{N+2P_{\mathrm{pad}}-K}{S}\right\rfloor+1
$$





$$
P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}
$$





$$
n_{\mathrm{flat}}=HWC
$$





$$
P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}
$$





$$
P_{\mathrm{pool}}=P_{\mathrm{flatten}}=0
$$



**Yaad rakhne ka cue:** Ek layer ka output, next layer ka input; shape aur parameters alag count.

**Stable softmax.** Exponentiate each logit and divide by the sum. Subtracting the largest logit first keeps the exponentials numerically manageable and leaves the probabilities unchanged.

zⱼ = class j score; M = largest score; pⱼ = class j probability. All probabilities sum to 1.



$$
M=\max_j z_j
$$





$$
p_j=\frac{e^{z_j-M}}{\sum_k e^{z_k-M}}
$$



**Yaad rakhne ka cue:** Largest score ghatao, exp nikalo, total se sabko divide karo.

**Categorical cross-entropy.** For mutually exclusive classes with a one-hot target, all zero-target terms disappear. The loss is the negative log of the probability assigned to the true class.

yⱼ = target entry; pⱼ = predicted probability; c = index of the true class. Use natural logarithms unless another base is specified.



$$
L=-\sum_j y_j\ln p_j
$$





$$
L=-\ln p_c
$$



**Yaad rakhne ka cue:** One-hot mein sirf correct class ki probability ka minus ln.

### Solution — step by step

### Formulas used

Separate shape from learned parameter count. H, W and C mean height, width and channels. K, P and S mean filter size, padding per side and stride.



$$
N_{\mathrm{out}}=\left\lfloor\frac{N+2P-K}{S}\right\rfloor+1
$$





$$
P_{\mathrm{conv}}=(K^2C_{\mathrm{in}}+1)C_{\mathrm{out}}
$$





$$
n_{\mathrm{flat}}=HWC,\qquad P_{\mathrm{dense}}=(n_{\mathrm{in}}+1)n_{\mathrm{out}}
$$



The +1 counts one bias per receiving filter or neuron. Pooling and flattening add no learned parameters. Softmax converts scores to probabilities; CCE uses the probability of the correct class.



$$
p_i=\frac{e^{z_i-c}}{\sum_j e^{z_j-c}},\quad c=\max_jz_j,\quad L=-\ln p_{\mathrm{correct}}
$$



Conv size = (8+2−3)+1 = 8; output 8 × 8 × 2; parameters = (3 × 3 × 3+1) × 2 = 56. Pool 4 × 4 × 2; parameters 0. Flatten 32. Dense 3 parameters = (32+1) × 3 = 99. Total = **155**.

**Step 1 stabilize:**  subtract the largest logit 1 from every value: [-1, 0, -2]. This does not change softmax.

**Step 2 exponentiate:**  exp values = [0.367879441, 1, 0.135335283].

**Step 3 denominator:**  sum = 1.503214724.

**Step 4 divide each value by this same sum:**  probabilities = **[0.244728471, 0.665240956, 0.090030573]**. **Check:**  all are positive and their sum is 1 (apart from rounding); the largest logit gets the largest probability.

Correct class probability = 0.665240956; CCE = −ln(0.665240956) = **0.407605964**.