# Drill 30 Regularized objective

Subject: Deep Learning

Exam: TEE

Reader: https://deep-learning-study-room.vercel.app/#tee/read/17/tee-c17-s30

Canonical: https://deep-learning-study-room.vercel.app/read/dl/tee/section/tee-c17-s30/

## Question

**Question.** Data loss = 0.4, w = [−2,1,0.5], λ = 0.1. Calculate L1 objective and L2 objective using (λ/2)Σw².

## Answer — step by step

### Given

Data loss = 0.4; weights = [−2,1,0.5]; λ = 0.1.

For L2, the question includes the factor ½.

### To find

Find each penalty and the total L1/L2 objective.

### Formula — which one and why

**L1 penalty and gradient.** Add absolute weight magnitudes to penalize large weights. Away from zero, the penalty derivative depends on the weight’s sign.

λ (lambda) = penalty strength; J\_data = data loss; sign(w) is +1 for positive and −1 for negative weights. At zero a subgradient convention is needed.



$$
J=J_{\mathrm{data}}+\lambda\sum_i|w_i|
$$





$$
g_i=g_{\mathrm{data},i}+\lambda\operatorname{sign}(w_i)
$$



**Yaad rakhne ka cue:** L1 absolute values leta hai; gradient mein sign aata hai.

**L2 penalty with a factor of one-half.** Square each weight, sum and multiply by λ/2. The one-half cancels the factor 2 when differentiating, so the extra gradient is λw.

λ = penalty strength; w = old weight; J\_data and g\_data = data loss and its gradient.



$$
J=J_{\mathrm{data}}+\frac\lambda2\sum_iw_i^2
$$





$$
g_i=g_{\mathrm{data},i}+\lambda w_i
$$





$$
w_i^{\mathrm{new}}=w_i-\eta g_i
$$



**Yaad rakhne ka cue:** Half wala L2 derivative mein lambda × weight deta hai.

### Solution — step by step

### Meaning and formulas

The data loss measures prediction error. A regularization penalty adds a cost for large weights. The objective J is the total being minimized. λ sets the strength of the penalty.



$$
J_{L1}=L_{\mathrm{data}}+\lambda\sum_i|w_i|
$$





$$
J_{L2}=L_{\mathrm{data}}+\frac\lambda2\sum_iw_i^2
$$



Absolute value means ignore the sign. Squaring means multiply the number by itself. These are different operations.

**Step 1: Part A: L1 objective.**



$$
|-2|=2,\quad |1|=1,\quad |0.5|=0.5
$$





$$
\sum_i|w_i|=2+1+0.5=3.5
$$





$$
\text{L1 penalty}=0.1\times3.5=0.35
$$





$$
J_{L1}=0.4+0.35=0.75
$$



**Step 2: Part B: L2 objective with the given half.**



$$
(-2)^2=4,\quad 1^2=1,\quad 0.5^2=0.25
$$





$$
\sum_iw_i^2=4+1+0.25=5.25
$$





$$
\frac\lambda2=\frac{0.1}2=0.05
$$





$$
\text{L2 penalty}=0.05\times5.25=0.2625
$$





$$
J_{L2}=0.4+0.2625=0.6625
$$



**Final answer:** L1 objective 0.75; L2 objective 0.6625. These are total objectives, not just penalty values.