Why position matters — permutation-invariant pooling

Background

Self-attention (without positions) treats a sequence as a set: reorder the tokens and the output is unchanged. A mean over positions is the simplest example of such a permutation-invariant operation — proof that order is invisible until you add positional information.

Problem statement

Implement mean_pool(X) returning the mean over the sequence (row) axis.

Input

• X — array-like of shape (L, d).

Output

Returns an np.ndarray of shape (d,): the per-feature mean over the L positions.

Examples

Example 1

Input:  X = [[1, 2], [3, 4]]
Output: [2, 3]

Explanation: column means of [1,3] and [2,4].

Example 2 — order doesn't matter

Input:  X = [[3, 4], [1, 2]]   (rows swapped)
Output: [2, 3]                  (same as above)

Constraints

• Average over the position axis (axis=0), giving shape (d,).
• The result must be identical for any row permutation of X.

Notes

• This invariance is exactly the problem positional encodings solve: with it, "dog bites man" and "man bites dog" would pool to the same vector.