Fuse token + positional embeddings

Background

The transformer's input is the element-wise sum of token embeddings and positional encodings — both live in $\mathbb{R}^{d_{\text{model}}}$, so the sum stays the same width:

$$\mathbf{z}_{\text{pos}} = \mathbf{x}_{\text{pos}} + \mathbf{p}_{\text{pos}}$$

This Z is what flows into the first block; only later do linear maps build Q, K, V from it.

Problem statement

Implement add_positional(token_emb, pos_emb) returning their element-wise sum.

Input

• token_emb — array-like of shape (L, d): token embeddings.
• pos_emb — array-like of shape (L, d): positional encodings.

Output

Returns an np.ndarray of shape (L, d): token_emb + pos_emb.

Examples

Example 1

Input:  token_emb = [[1, 2], [3, 4]], pos_emb = [[0.1, 0.1], [0.2, 0.2]]
Output: [[1.1, 2.1], [3.2, 4.2]]

Constraints

• Add elementwise; the shape is unchanged.
• Return an np.ndarray.

Notes

• Addition (not concatenation) keeps the dimension at d_model, so a single residual stream carries both "what token" and "where" — the cheap, elegant choice from the paper.