BCE from logits — the stable sigmoid + BCE pairing

Background

Sigmoid + BCE is a designed pairing. Composing them naively — sigmoid, then log — can overflow for large logits. The fused, numerically stable form computes BCE straight from the logit $z$ (where $\hat{y} = \sigma(z)$):

$$L(z, y) = \max(z, 0) - z\,y + \log\!\bigl(1 + e^{-|z|}\bigr)$$

This equals $-[y\log\sigma(z) + (1-y)\log(1-\sigma(z))]$ but never overflows, because $e^{-|z|} \le 1$.

Problem statement

Implement bce_with_logits(z, y) returning the mean stable BCE over a batch of logits.

Input

• z — array-like of logits (pre-sigmoid scores), any real values.
• y — array-like of labels (0/1), same length.

Output

Returns a Python float: the mean BCE, computed stably from the logits.

Examples

Example 1 — agrees with the sigmoid-then-BCE route

Input:  z = [2, -1, 0.5], y = [1, 0, 1]
Output: 0.4564845...

Explanation: same value as applying sigmoid then plain BCE, but overflow-safe.

Example 2 — extreme logit stays finite

Input:  z = [100], y = [0]
Output: ~100.0

Explanation: predicting class 1 with huge confidence when the truth is 0 costs $\approx z$, and the formula returns it without inf.

Constraints

• Use the stable form max(z,0) - z*y + log(1 + exp(-|z|)) per element, then average.
• Must stay finite for large |z| (no overflow).
• Return a plain float.

Notes

• This is what BCEWithLogitsLoss / sigmoid_cross_entropy_with_logits do under the hood — and why you pass logits, not probabilities, to those functions.