From encoder–decoder to GPT · Prof · Prof by Lex AI

Curriculum Practice For Organizations Pricing

Start Your AI JourneyFree to start · No credit card required

Foundations Vision Sequences Transformers Agentic AI System Design Practice Interview

Transformers

Attention Is All You Need0/11

Problem with RNNs and LSTMs
Positional embeddings & positional encodings
Attention
Self-attention
Multi-headed attention
Cross-attention
The encoder stack
Encoder–decoder transformer
From encoder–decoder to GPT
Interview Readiness, Attention Is All You Need

Build and Train Your Own GPT-20/12

Problem with RNNs and LSTMs
Token Embeddings
Positional Embeddings / Positional Encodings
Attention + Multi-Head Attention
Causal Masking
Residual Connections
Layer Normalization
Feed Forward Neural Networks
Generation of Next Tokens
Decoder Only Transformer
Interview Readiness, Build and Train Your Own GPT-2

Courses/Transformers/Attention Is All You Need/From encoder–decoder to GPT

Reading progress

0% read

Course progress

0 / 23 lessons

Transformers

On this page

No sections yet.

Interview Readiness, Attention Is All You Need