Regularization

10 problems

• AdamW step (decoupled weight decay)Medium
• BatchNorm forward (train + eval modes)Medium
• Overfitting — read the learning curvesEasy
• Weight decay — the L2 penalty termEasy
• Dropout layer (forward & backward)Medium
• Early stopping on validation lossEasy
• Elastic-Net regression (gradient descent)Medium
• Exponential moving average of weightsEasy
• Label-smoothed cross-entropyMedium
• Ridge regression lossEasy