Faithfulness / Groundedness Score

Background

Production RAG systems cannot rely on the LLM "doing the right thing" — they have to measure it. Faithfulness (also called groundedness) is the standard scalar that asks: of the atomic claims the answer makes, what fraction is genuinely supported by the cited passages? It is the metric that separates an answer that reads plausible from one that is actually grounded. Tools like Ragas, TruLens, and DeepEval all compute a per-claim version of it.

Problem statement

Implement faithfulness(claim_scores, strict_threshold=None). Given a list of per-claim entailment scores in $[0, 1]$ (one score per atomic claim in the answer, produced upstream by a small NLI model or a structured LLM judge), return the mean as the faithfulness:

$$\text{faithfulness} = \frac{1}{N}\sum_{i=1}^{N}\text{entailment}(\text{claim}_i, \text{passages})$$

If strict_threshold is given, the call switches to strict mode: if any single claim's score falls below the threshold, the answer is treated as ungrounded and the function returns $0$. Strict mode is the production setting for high-stakes domains (medical, legal, financial) where a 95% mean is not enough — every claim has to clear the bar individually.

Input

• claim_scores — array-like of floats in $[0, 1]$ (one per atomic claim in the answer).
• strict_threshold — Optional[float] in $[0, 1]$. None (default) means the standard mean; if given, every claim must score $\ge$ this threshold or the result collapses to $0$.

Output

Returns a Python float in $[0, 1]$ — the faithfulness score. The empty-input case returns 0.0 (an answer with zero atomic claims is treated as ungrounded, not perfectly faithful).

Examples

Example 1 — standard mean

Input:  claim_scores = [0.9, 0.8, 1.0]
Output: 0.9

Explanation: $\tfrac{0.9 + 0.8 + 1.0}{3} = 0.9$.

Example 2 — strict mode tolerates only claims at or above the threshold

Input:  claim_scores = [0.95, 0.6, 1.0], strict_threshold = 0.7
Output: 0.0

Explanation: the middle claim is below the strict threshold of $0.7$, so the whole answer is rejected.

Example 3 — strict mode passes when every claim clears the bar

Input:  claim_scores = [0.95, 0.8, 1.0], strict_threshold = 0.7
Output: 0.9166666666666666

Explanation: every claim is at or above $0.7$, so the function falls back to the mean: $\tfrac{0.95 + 0.8 + 1.0}{3} \approx 0.9167$.

Constraints

• Scores live in $[0, 1]$; the result lies in $[0, 1]$.
• Empty input ($N = 0$) returns 0.0 — an unsupported answer, not a vacuously perfect one.
• Return a plain Python float, not a 0-d numpy scalar.

Notes

• Why per-claim, not per-answer. A single end-to-end "is the answer grounded?" score conflates retrieval miss with generation hallucination with answer-quality drift. Per-claim entailment localises the failure to the specific unsupported sentence, which is what lets a system either withhold the response, annotate the unsupported claim, or trigger a refusal.
• Calibration. The entailment scores feeding this function come from a separate NLI step that has to be calibrated against a human gold set; ungrounded scores will produce ungrounded faithfulness numbers. That calibration is upstream of this function but is the load-bearing piece of any production RAG-eval stack.