GPTConfig.head_size with the divisibility check

Background

GPT-2's config exposes a derived head_size — the per-head dimension — computed from the embedding width and the number of heads:

$$\text{head\_size} = \frac{n\_embd}{n\_head}$$

It must divide evenly (the heads concatenate back to n_embd), so the config asserts n_embd % n_head == 0. For GPT-2 small: 768 / 12 = 64.

Problem statement

Implement head_size(n_embd, n_head) returning n_embd // n_head, but raising ValueError when n_head does not divide n_embd.

Input

• n_embd — int, embedding/hidden dimension.
• n_head — int, number of attention heads.

Output

Returns an int n_embd // n_head; raises ValueError if n_embd % n_head != 0.

Examples

Example 1 — GPT-2 small

Input:  n_embd = 768, n_head = 12
Output: 64

Example 2 — invalid

Input:  n_embd = 768, n_head = 5
Raises: ValueError

Constraints

• Validate n_embd % n_head == 0; otherwise raise ValueError(...).
• Return the integer head size.

Notes

• This mirrors the @property head_size on GPTConfig — one number that every attention module reads, so the check belongs here.