One-hot encode a token sequence

Background

The simplest way to turn token IDs into vectors is one-hot encoding: each ID becomes a length-$|\mathcal{V}|$ vector that is all zeros except a single 1 at the token's index.

$$\text{onehot}(i) \in \{0,1\}^{|\mathcal{V}|}, \quad \text{onehot}(i)_j = \mathbb{1}[j = i]$$

Problem statement

Implement one_hot(ids, vocab_size) returning the one-hot matrix for a sequence of token IDs.

Input

• ids — array-like of T integer token IDs in $[0, \text{vocab\_size})$.
• vocab_size — int, the vocabulary size $|\mathcal{V}|$.

Output

Returns an np.ndarray of shape (T, vocab_size): row t is the one-hot vector of ids[t].

Examples

Example 1

Input:  ids = [0, 2], vocab_size = 3
Output: [[1, 0, 0], [0, 0, 1]]

Example 2 — single token

Input:  ids = [1], vocab_size = 2
Output: [[0, 1]]

Constraints

• Each row has exactly one 1, at the column equal to that token's ID.
• Return shape (T, vocab_size).

Notes

• One-hot vectors are huge (size $|\mathcal{V}|$) and mutually orthogonal — they carry no notion of similarity, which is exactly why learned embeddings replaced them as model inputs.