QA Token F1 / Exact Match

Background

Token-level F1 and Exact Match (EM) are the canonical short-answer QA metrics (SQuAD-style). They tokenise both prediction and gold answer, lowercase, then compute the multiset overlap. F1 captures partial credit; EM is binary on the normalised strings.

Problem statement

Implement qa_metrics(pred, gold). Lowercase + whitespace-tokenise both strings. Let $C$ be the count of overlapping tokens (multiset intersection). Compute:

$$P = \frac{C}{|pred|},\quad R = \frac{C}{|gold|},\quad F_1 = \frac{2 P R}{P + R},\quad \text{EM} = \mathbb{1}[pred = gold]$$

after the same normalisation.

Input

• pred — str (or None/empty), the predicted answer.
• gold — str (or None/empty), the gold answer.

Output

Returns a tuple (f1: float, em: float) — both in $[0, 1]$. em is $0.0$ or $1.0$.

Examples

Example 1 — exact match

Input:  pred="the cat sat", gold="the cat sat"
Output: (1.0, 1.0)

Example 2 — partial overlap

Input:  pred="the cat ran", gold="the cat sat"
Output: (~0.6667, 0.0)

Explanation: overlap = {the, cat}; $P = R = 2/3$, $F_1 = 2/3$. EM fails.

Example 3 — zero overlap

Input:  pred="xyz", gold="abc"
Output: (0.0, 0.0)

Example 4 — both empty

Input:  pred="", gold=""
Output: (1.0, 1.0)

Constraints

• Tokenisation is whitespace-only after lowercasing — no punctuation stripping in this simplified version.
• Both empty → F1 = 1, EM = 1 (vacuous agreement).
• One empty, the other not → F1 = 0.

Notes

• SQuAD normalisation. Real SQuAD eval also strips articles ("a", "an", "the") and punctuation. Out of scope for this kernel.
• Multiset matters. Overlap on a multiset (tokens with counts) penalises repeated tokens correctly; on a set it would over-credit short answers.