Logistic regression — the BCE gradient (error × feature)

Background

Differentiating binary cross-entropy looks scary, but the sigmoid factor $\hat{y}(1-\hat{y})$ cancels the $1/\hat{y}$ from the log and the cross terms vanish, leaving a strikingly simple gradient:

$$\frac{\partial J}{\partial \theta_j} = \frac{1}{m}\sum_{i=1}^{m}(\hat{y}_i - y_i)\,x_{ij} \qquad\Longrightarrow\qquad \nabla_{\boldsymbol\theta} J = \frac{1}{m}X^\top(\hat{\mathbf{y}} - \mathbf{y})$$

Read it as error × feature: $(\hat{y}_i - y_i)$ is the prediction error, and multiplying by $x_{ij}$ says "nudge $\theta_j$ in proportion to how much feature $j$ drove that error." It is the same shape as the linear-regression gradient — only $\hat{y}=\sigma(z)$ changed.

Problem statement

Implement bce_gradient(X, y, y_hat) returning $\frac{1}{m}X^\top(\hat{\mathbf{y}} - \mathbf{y})$.

Input

• X — array-like of shape (m, n): one feature row per example.
• y — array-like of m true labels (0/1).
• y_hat — array-like of m predicted probabilities.

Output

Returns an np.ndarray of shape (n,): the gradient, one component per feature.

Examples

Example 1 — zero error, zero gradient

Input:  X = [[1, 2], [3, 4]], y = [1, 0], y_hat = [1, 0]
Output: [0. 0.]

Explanation: predictions match labels, so $\hat{y}-y=0$ and the gradient vanishes.

Example 2

Input:  X = [[1, 2], [1, 3]], y = [1, 0], y_hat = [0.5, 0.5]
Output: [0.   0.25]

Explanation: errors are $[-0.5, 0.5]$; $X^\top(\hat{y}-y) = [\,0,\ 0.5\,]$, divided by $m=2$ gives $[0, 0.25]$.

Constraints

• Use the error $\hat{\mathbf{y}} - \mathbf{y}$ (predicted minus true), then project with $X^\top$ and divide by $m$.
• Return an np.ndarray of shape (n,).

Notes

• This is the gradient you plug into the update $\boldsymbol\theta \leftarrow \boldsymbol\theta - \eta\,\nabla_{\boldsymbol\theta}J$ to train logistic regression — structurally identical to the linear-regression step from Lesson 1.