Refusal / Confidence Gate

Background

RAG and LLM systems should refuse when retrieval confidence is below a threshold rather than hallucinate. The simplest gate compares the top retrieval / model confidence to a configured threshold and returns a refusal message if below. Standard last-mile guardrail in production RAG.

Problem statement

Implement confidence_gate(score, answer, threshold=0.5, refusal="I don't have enough information to answer."). Return refusal if score < threshold; return answer otherwise. Inclusive at the threshold (≥).

Input

• score — float, the top retrieval or model confidence.
• answer — str, the candidate answer.
• threshold — float, the cutoff (default 0.5).
• refusal — str, the refusal text (default: standard message).

Output

Returns str.

Examples

Example 1 — high confidence

Input:  score=0.9, answer="yes"
Output: "yes"

Example 2 — low confidence

Input:  score=0.2, answer="yes"
Output: refusal message

Example 3 — exactly at threshold passes

Input:  score=0.5, threshold=0.5
Output: answer

Example 4 — custom refusal

Input:  score=0.1, refusal="REFUSE"
Output: "REFUSE"

Constraints

• Inclusive at threshold: score >= threshold returns the answer.
• score and threshold are real-valued floats.
• Returns the refusal string verbatim (no formatting).

Notes

• Calibrated scores matter. The threshold is meaningful only if the score is calibrated. Pair this with Platt or isotonic calibration upstream.
• Multi-stage gating. Production systems chain multiple gates: retrieval confidence, generation confidence, citation accuracy. Any single failure trips refusal.