Cross-attention score shape (target × source)

Background

In cross-attention, queries are target positions and keys are source positions, so the score matrix $QK^\top$ is (target_len × source_len) — rectangular, not square:

$$QK^\top \in \mathbb{R}^{n \times m}$$

with n = target_len, m = source_len. Softmax runs over the source axis, giving each target step a distribution over the input.

Problem statement

Implement cross_score_shape(target_len, source_len) returning the shape of the cross-attention score matrix.

Input

• target_len — int, number of decoder (query) positions.
• source_len — int, number of encoder (key) positions.

Output

Returns a tuple (target_len, source_len).

Examples

Example 1

Input:  target_len = 3, source_len = 5
Output: (3, 5)

Explanation: 3 target queries each scoring all 5 source keys.

Constraints

• Return (target_len, source_len) as a tuple.

Notes

• This rectangular shape is the difference from self-attention's square (n × n): cross-attention cost is $O(n\cdot m)$, bilinear in the two lengths.