Sequential Test (mSPRT-Style Always-Valid p-Value)

Background

Standard A/B tests with peeking inflate Type-I error. mSPRT-style always-valid p-values let an experimenter check the running result at any time without the false-positive rate exceeding $\alpha$. The kernel: at each step, compute a $z$-statistic on the running differences; map to a bounded $p$ via $\min(1, e^{-z^2/2})$.

Problem statement

Implement running_avalp(differences) returning a list of always-valid $p$-values, one per step:

$$\bar x_n = \tfrac1n\sum d_i,\quad s_n^2 = \tfrac1{n-1}\sum (d_i - \bar x_n)^2,\quad z_n = \frac{|\bar x_n|}{s_n / \sqrt n},\quad p_n = \min\!\Bigl(1,\, e^{-z_n^2 / 2}\Bigr)$$

Input

• differences — list[float], the per-step paired differences (treatment − control).

Output

Returns list[float] of length len(differences), each in $[0, 1]$.

Examples

Example 1 — all-zero stream

Input:  [0.0, 0.0, ..., 0.0]
Output: all 1.0

Example 2 — strong signal eventually crosses 0.05

Input:  [1.0] * 100
Output: at some n, p < 0.05; stays below thereafter

Example 3 — n=1 special case

Input:  [0.5]
Output: [1.0]   # no variance estimate yet, return 1

Constraints

• $n < 2$ → return 1.0 for that step.
• Empty input → empty list.
• All p-values bounded in $[0, 1]$.

Notes

• Why this is "always valid". Peeking-safe means $\Pr[\exists n : p_n < \alpha\,\mid H_0] \le \alpha$ — the false-positive rate at any stopping time is bounded. The exponential bound comes from mSPRT theory.
• vs fixed-horizon. Always-valid $p$-values are typically 1.5–3× higher than fixed-horizon $p$-values at the planned sample size. Tradeoff: peeking flexibility for some power.