One-hot tokens are all equidistant

Background

The other failure of one-hot inputs: they carry no similarity structure. Any two distinct one-hot vectors differ in exactly two coordinates (a 1 becomes 0, a 0 becomes 1), so their Euclidean distance is always:

$$\lVert e_i - e_j \rVert_2 = \sqrt{2} \quad (i \ne j), \qquad 0 \quad (i = j)$$

"cat" is exactly as far from "dog" as from "subpoena". Learned embeddings fix this by letting similar tokens sit closer.

Problem statement

Implement onehot_distance(i, j) returning the Euclidean distance between the one-hot vectors for token ids i and j.

Input

• i, j — int token ids.

Output

Returns a float: 0.0 if i == j, else sqrt(2).

Examples

Example 1 — same token

Input:  i = 5, j = 5
Output: 0.0

Example 2 — different tokens

Input:  i = 5, j = 100
Output: 1.4142135623730951

Constraints

• i == j → 0.0; otherwise sqrt(2).
• The distance does not depend on how far apart the ids are.

Notes

• This flat geometry is exactly what a learned embedding throws away in favour of meaningful distances — synonyms close, unrelated tokens far.