Encoder–decoder transformer · Prof · Prof by Lex AI

Curriculum Practice For Organizations Pricing

Sign In Start Learning

Foundations Vision Sequences Transformers Agentic AI System Design Practice Interview

Transformers

Attention Is All You Need0/11

Problem with RNNs and LSTMs
Positional embeddings & positional encodings
Attention
Self-attention
Multi-headed attention
Cross-attention
The encoder stack
Encoder–decoder transformer
From encoder–decoder to GPT
Interview Readiness, Attention Is All You Need

Build and Train Your Own GPT-20/12

Problem with RNNs and LSTMs
Token Embeddings
Positional Embeddings / Positional Encodings
Attention + Multi-Head Attention
Causal Masking
Residual Connections
Layer Normalization
Feed Forward Neural Networks
Generation of Next Tokens
Decoder Only Transformer
Interview Readiness, Build and Train Your Own GPT-2