Vanishing gradients through sigmoid layers

Background

Backprop multiplies a sigmoid slope $\sigma'(z) = a(1-a)$ at every layer. Since $a(1-a) \le 0.25$ always, multiplying many of them together drives the product toward zero — the vanishing gradient problem that makes deep sigmoid stacks hard to train.

Problem statement

Implement gradient_product(activations) returning the product of the sigmoid slopes $\prod_\ell a_\ell(1-a_\ell)$ across a chain of layer activations.

Input

• activations — array-like of sigmoid activations $a_\ell$ (one representative value per layer), each in $(0,1)$.

Output

Returns a Python float: $\prod_\ell a_\ell(1-a_\ell)$.

Examples

Example 1 — one layer at its steepest

Input:  activations = [0.5]
Output: 0.25

Explanation: $0.5\times0.5 = 0.25$, the maximum possible per-layer slope.

Example 2 — three layers

Input:  activations = [0.5, 0.5, 0.5]
Output: 0.015625

Explanation: $0.25^3$ — already 16× smaller after three layers.

Constraints

• Compute $a(1-a)$ per layer, then take the product.
• Return a plain float.

Notes

• Because every factor is $\le 0.25$, an $L$-layer sigmoid path attenuates the gradient by at least $4^L$ — which is why ReLU (slope 1 for $z>0$) replaced sigmoid between hidden layers.