ReLU — find the dead neurons

Background

A ReLU unit outputs $\max(0, z)$. If a neuron's pre-activation $z$ is $\le 0$ for every input it ever sees, its output is always $0$ and its gradient is always $0$ — it can never learn. This is the dead neuron problem.

Given a batch of pre-activations (one row per example, one column per neuron), a neuron is dead when its whole column is $\le 0$.

Problem statement

Implement dead_neurons(Z) returning a boolean mask flagging which neurons (columns) are dead across the batch.

Input

• Z — array-like of shape (batch, units): pre-activations before ReLU.

Output

Returns a boolean np.ndarray of length units: True where that neuron's pre-activation is $\le 0$ for all rows.

Examples

Example 1

Input:  Z = [[-1, 2, -3], [-4, 5, -6]]
Output: [ True False  True]

Explanation: column 0 is $\{-1,-4\}$ (all $\le 0$ → dead), column 1 has a positive value (alive), column 2 is all negative (dead).

Example 2 — a zero column is dead

Input:  Z = [[0, 1], [0, 2]]
Output: [ True False]

Explanation: ReLU(0) = 0 with zero gradient, so an all-zero column counts as dead.

Constraints

• A neuron is dead iff the maximum of its column is $\le 0$.
• Return a boolean np.ndarray of length units.

Notes

• Dead neurons motivate Leaky ReLU / GELU, which keep a small gradient for $z \le 0$ so units can recover.