Neural Networks

95 problems

• Affine coupling layer (normalizing flow)Medium
• Teacher forcing — next-token targetsMedium
• Pretraining objective per familyEasy
• Autoencoder forward passEasy
• Basic autograd operationsMedium
• Binary Quantization + Hamming RetrievalMedium
• Conv2D backward (dx, dW)Hard
• Linear forward (Wx + b)Easy
• ReLU forward + backwardEasy
• Sigmoid forward + backwardEasy
• MSE loss + backwardEasy
• Linear backward (chain rule)Medium
• SGD step (in place)Easy
• Train a 2-layer MLP on XORHard
• Warmup + cosine decay LR scheduleEasy
• Cross-entropy gradientMedium
• Convolution — slide a 1D filter (weight sharing)Medium
• Images as tensors — the shapeEasy
• Edge models — magnitude pruningEasy
• Edge models — measure sparsityEasy
• DDPM Reverse (Denoise) StepMedium
• Dense block with 2D convolutionsMedium
• Diffusion forward processMedium
• Backprop — all four gradients for one hidden layerMedium
• From scratch — the batched forward passMedium
• BCE from logits — the stable sigmoid + BCE pairingMedium
• End-to-end binary classifier (ReLU hidden, sigmoid output)Medium
• Binary output head — probability and decisionEasy
• BCE — clip the probabilities so log stays finiteEasy
• Forward pass — arbitrary depthMedium
• Dense layer — count the parametersEasy
• Dense layer — sigmoid forward passEasy
• Overfitting — read the learning curvesEasy
• Ensembling — average models, then predictMedium
• Forward pass — cache the activationsEasy
• Hyperparameter grid-search sizeEasy
• From scratch — derive the parameter shapesEasy
• Parameter or hyperparameter?Easy
• Weight decay — the L2 penalty termEasy
• Why nonlinearity — collapsing two linear layersMedium
• Network — total trainable parametersEasy
• Neuron — one unit, three activationsEasy
• Neuron — the linear score z = wᵀx + bEasy
• Backprop — the output deltaEasy
• Perceptron — the hard threshold unitEasy
• Perceptron — the learning ruleMedium
• Loss — pick the better modelEasy
• Output layer — predict the classEasy
• ReLU — find the dead neuronsEasy
• Dense layer with ReLU (the production hidden default)Easy
• Scaling laws — estimate the power-law exponentMedium
• Sigmoid derivative from the activationEasy
• Softmax with temperatureEasy
• From scratch — does this init break symmetry?Easy
• Network — the two-layer forward passMedium
• Vanishing gradients through sigmoid layersEasy
• Backprop — the weight gradient is an outer productEasy
• Dropout layer (forward & backward)Medium
• Early stopping on validation lossEasy
• Exponential moving average of weightsEasy
• Exponential LR schedulerEasy
• GCN layer (message passing)Medium
• GELU backward (tanh approximation)Medium
• Parameter breakdown of a GPT-2 blockMedium
• The 4× MLP hidden dimensionEasy
• Where loss starts: log(V)Easy
• Depth-aware init for residual projectionsEasy
• Gradient checkpointing forwardEasy
• Clip gradients by global L2 normEasy
• Group normalizationMedium
• Hard Negative Mining for RetrieverMedium
• He weight initializationEasy
• HNSW Greedy Graph SearchHard
• IVF (Inverted File) SearchMedium
• Label-Noise Detection (Confident Learning)Hard
• Leaky ReLU activationEasy
• Learning-rate range testMedium
• Log-softmaxEasy
• Matmul backwardMedium
• MIPS -> L2 ReductionMedium
• Neural ODE forward EulerMedium
• Product Quantization (PQ) Encode + ADCHard
• Random-Hyperplane LSH (Cosine)Medium
• Residual block with shortcutEasy
• SimpleRNN forward — outputs over a sequenceMedium
• In-Batch Sampled Softmax + logQ CorrectionHard
• SELU activationEasy
• BPTT — the gradient-norm trajectoryMedium
• Vanishing or exploding? Classify the regimeEasy
• BPTT — the tanh backward factorEasy
• Softmax backwardMedium
• Softmax from scratchEasy
• Step LR schedulerEasy
• Swish / SiLU activationEasy
• VAE ELBO lossMedium