RNN

26 problems

• Per-layer complexity — attention vs recurrenceMedium
• Token-to-token path lengthEasy
• Sequential steps — serial vs parallelEasy
• Beam search decodingMedium
• Fixed vs growing memory (the RNN bottleneck)Easy
• Vanilla RNN cell forwardMedium
• SimpleRNN forward — outputs over a sequenceMedium
• RNN output — project the hidden stateEasy
• RNN parameters — weight sharing across timeEasy
• Loss over time — sum the per-step lossesEasy
• Unroll the recurrence over a sequenceMedium
• BPTT — the gradient-norm trajectoryMedium
• Build a vocabularyEasy
• Encode a sentence (with UNK)Easy
• Vanishing or exploding? Classify the regimeEasy
• Many-to-one — take the last timestepEasy
• History beats a single point — next-step predictionEasy
• One-hot encode a token sequenceEasy
• The no-memory baselineEasy
• Prepare RNN data — scale, window, reshapeMedium
• nn.RNN — tensor shapes through the modelEasy
• Build sliding-window training pairsMedium
• BPTT — the tanh backward factorEasy
• Classify the sequence taskEasy
• Sequence padding & maskingEasy
• Viterbi algorithmMedium