Logistic regression — binary cross-entropy loss

Background

Taking the log of the dataset likelihood turns the product into a sum, and negating + averaging it gives the loss logistic regression actually minimises — binary cross-entropy (log loss):

$$J = -\frac{1}{m}\sum_{i=1}^{m}\Bigl[\,y_i \log \hat{y}_i + (1-y_i)\log(1-\hat{y}_i)\,\Bigr]$$

For $\hat{y}_i = \sigma(\mathbf{w}^\top\mathbf{x}_i + b)$ this $J$ is convex, so gradient descent reaches the single global optimum. It also punishes confident-and-wrong predictions hard: if $y=1$ but $\hat{y}\to 0$, $\log\hat{y}\to-\infty$.

Problem statement

Implement binary_cross_entropy(y, y_hat) returning the mean BCE over all examples.

Input

• y — array-like of true labels, each 0 or 1.
• y_hat — array-like of predicted probabilities in $(0, 1)$, same length.

Output

Returns a Python float: the mean binary cross-entropy.

Examples

Example 1

Input:  y = [1], y_hat = [0.5]
Output: 0.6931471805...

Explanation: $-\log(0.5) = \ln 2 \approx 0.6931$ — the loss of a coin-flip prediction.

Example 2

Input:  y = [1, 0], y_hat = [0.9, 0.1]
Output: 0.1053605156...

Explanation: both predictions are confident and correct, so each term is $-\log(0.9)$ and the mean is $\approx 0.1054$.

Constraints

• Average over the $m$ examples (the $1/m$ factor).
• Assume y_hat is strictly in $(0, 1)$ — no clipping required here.
• Return a plain float.

Notes

• Lower is better; a perfect, confident model drives $J \to 0$. This is the negative log-likelihood of a Bernoulli model — minimising it maximises the likelihood of the labels.