LayerNorm statistics — per token, over features

Background

LayerNorm normalises each token independently using statistics computed over its feature axis only (no batch, no time). For a token vector x ∈ ℝ^C:

$$\mu = \frac{1}{C}\sum_c x_c, \qquad \sigma^2 = \frac{1}{C}\sum_c (x_c - \mu)^2$$

Both are scalars per token — one mean and one variance per row.

Problem statement

Implement layernorm_stats(x) returning the per-token mean and variance over the last axis.

Input

• x — array of shape (T, C) (one token per row).

Output

Returns (mean, var), each an np.ndarray of shape (T,): the mean and (population) variance over the C features of each row.

Examples

Example 1

Input:  x = [[1, 2, 3, 4]]
Output: mean = [2.5], var = [1.25]

Explanation: mean 2.5; variance mean((x-2.5)²) = 1.25.

Constraints

• Reduce over the last axis (features).
• Use population variance (divide by C, i.e. ddof=0).
• Each is shape (T,).

Notes

• Because the stats use only a token's own features, LayerNorm behaves identically in training and one-token-at-a-time generation — unlike BatchNorm.