Self or cross attention?

Background

The only thing separating self-attention from cross-attention is where Q, K, V come from:

• Self-attention: Q and the K/V both come from the same sequence.
• Cross-attention: Q comes from one sequence (the decoder), K and V from another (the encoder).

The scaled dot-product math is identical either way.

Problem statement

Implement attention_kind(q_source, kv_source) returning the attention type.

Input

• q_source — string naming where Q comes from.
• kv_source — string naming where K and V come from.

Output

Returns "self" if the two sources are the same, else "cross".

Examples

Example 1 — decoder queries encoder memory

Input:  q_source = "decoder", kv_source = "encoder"
Output: "cross"

Example 2 — encoder self-attention

Input:  q_source = "encoder", kv_source = "encoder"
Output: "self"

Constraints

• Same source → "self"; different → "cross".
• Return the exact string.

Notes

• "Multi-head" is orthogonal: you can have multi-head self-attention or multi-head cross-attention — that knob is about how many parallel heads, not about the sources.