Assemble the full GPT-2 parameter count

Background

The whole model is: embeddings + n_layer transformer blocks + one final LayerNorm. The LM head is tied to the token embedding, so it adds nothing.

• Embeddings: vocab_size·C + block_size·C (token + positional).
• Per block: 12C² + 4C (attn 4C², MLP 8C², two LNs 4C).
• Final LN: 2C.
• LM head: 0 (weight-tied).

For GPT-2 small this lands at ~124M.

Problem statement

Implement gpt_total_params(vocab_size, block_size, n_embd, n_layer) returning the total parameter count.

Input

• vocab_size, block_size, n_embd (C), n_layer (L).

Output

Returns an int: embeddings + L·(12C² + 4C) + 2C.

Examples

Example 1 — GPT-2 small

Input:  vocab_size = 50257, block_size = 1024, n_embd = 768, n_layer = 12
Output: 124356864

Example 2 — tiny config

Input:  vocab_size = 10, block_size = 4, n_embd = 2, n_layer = 1
Output: 88

Explanation: embeddings 28 + block 56 + final LN 4.

Constraints

• embeddings = (vocab_size + block_size) * n_embd.
• per_block = 12*n_embd**2 + 4*n_embd; blocks = n_layer * per_block.
• final_ln = 2*n_embd; LM head adds 0 (tied).
• total = embeddings + blocks + final_ln.

Notes

• Weight tying is why the output projection contributes nothing — the same (V, C) matrix serves embedding and unembedding.