Weight decay — the L2 penalty term

Background

L2 regularization (weight decay) discourages large weights by adding a penalty to the loss:

$$L_{\text{total}} = L_{\text{data}} + \lambda \lVert W \rVert^2 = L_{\text{data}} + \lambda \sum_{j} W_j^2$$

Smaller weights mean smoother, simpler decision boundaries that generalise better. Here you compute just the penalty term.

Problem statement

Implement l2_penalty(W, lam) returning $\lambda \sum_j W_j^2$ for a weight array of any shape.

Input

• W — array-like of weights (vector or matrix).
• lam — float, the regularization strength $\lambda$.

Output

Returns a Python float: $\lambda$ times the sum of squared weights.

Examples

Example 1

Input:  W = [[1, 2], [3, 4]], lam = 0.1
Output: 3.0

Explanation: $0.1\,(1+4+9+16) = 0.1\times30 = 3.0$.

Example 2 — no regularization

Input:  W = [3, 4], lam = 0.0
Output: 0.0

Constraints

• Sum the squares of all entries of W, then multiply by lam.
• Return a plain float.

Notes

• Biases are usually left out of weight decay; here we apply it to whatever W you pass. The gradient of this term is $2\lambda W$, which is what "decays" the weights each step.