#13PASS or FAIL?EasyEvaluation MetricsAgentic AI
PASS or FAIL?
Background
The Critic audits an answer for two kinds of problems: GAPs (sub-questions with no useful data the answer glosses over) and UNSUPPORTED claims (statements not backed by any finding). The verdict rule:
FAILif any gap or unsupported claim was found, otherwisePASS.
Problem statement
Implement critic_verdict(gaps, unsupported) returning the verdict string.
Input
gaps— list of gap descriptions (possibly empty).unsupported— list of unsupported-claim descriptions (possibly empty).
Output
Returns "FAIL" if either list is non-empty, else "PASS".
Examples
Example 1 — clean
Input: gaps = [], unsupported = []
Output: "PASS"
Example 2 — an unsupported claim
Input: gaps = [], unsupported = ["adopted in StarCoder and CodeLlama"]
Output: "FAIL"
Constraints
- Any non-empty list →
"FAIL". - Both empty →
"PASS".
Notes
- The verdict is the machine-readable signal an orchestrator branches on — diagnosis the next lesson will finally act on.
Python
Loading...
▶ Run executes the 3 visible sample tests below in your browser. Submit runs the full suite — including hidden tests — on the server for an official verdict.
- •Reference example: no issues -> PASS
- •Sample: an unsupported claim -> FAIL
- •Example: a gap -> FAIL