Jensen-Shannon Divergence (Drift)

Background

Jensen-Shannon divergence is a symmetric, bounded ($[0, 1]$ when using $\log_2$) divergence between two probability distributions - the variance-reduction sibling of KL. As a drift detector it has nicer properties than raw KL: symmetric, finite even when one distribution has zeros, and interpretable as a similarity.

Problem statement

Implement js_divergence_drift(expected, actual, n_bins=10, eps=1e-10) returning the JS divergence between the binned histograms:

$$\text{JS}(P \| Q) \;=\; \tfrac{1}{2} \text{KL}(P \| M) + \tfrac{1}{2} \text{KL}(Q \| M),\quad M = \tfrac{1}{2}(P + Q)$$

with $\log_2$ so the result lies in $[0, 1]$.

Input

• expected, actual - array-likes of floats.
• n_bins, eps - same convention as PSI.

Output

• float in $[0, 1]$.

Examples

Input: identical samples -> JS ~ 0
Input: disjoint shifted samples -> JS approaching 1

Constraints

• Equal-width bins on the expected sample's range, with clipping.
• $\log_2$ so the bound is $[0, 1]$.
• Returns $0$ if expected has zero range.

Notes

• JS vs KL. KL goes to $\infty$ when $P(x) > 0$ but $Q(x) = 0$. JS averages with $M$ so it's always finite. That makes it monitoring-friendly.