Forward pass — cache the activations

Background

The forward pass through the canonical $i \to h \to o$ net computes two activations:

$$\mathbf{a}_1 = \sigma(W_1\mathbf{x} + \mathbf{b}_1), \qquad \mathbf{a}_2 = \sigma(W_2\mathbf{a}_1 + \mathbf{b}_2)$$

For inference you only need $\mathbf{a}_2$. But to train (Lesson 6, backprop) you must keep the hidden activation $\mathbf{a}_1$ too — the backward pass reuses it. So a training-ready forward pass returns both.

Problem statement

Implement forward_cache(x, W1, b1, W2, b2) returning the tuple (a1, a2).

Input

• x — input vector, length i.
• W1 (h, i), b1 (h,) — input → hidden.
• W2 (o, h), b2 (o,) — hidden → output.

Output

Returns (a1, a2): the hidden activation (np.ndarray shape (h,)) and the prediction (np.ndarray shape (o,)), both in $(0,1)$.

Examples

Example 1

Input:  x = [0, 0], W1 = [[1, 0], [0, 1]], b1 = [0, 0], W2 = [[1, 1]], b2 = [0]
Output: a1 = [0.5, 0.5],  a2 = [0.73105858]

Explanation: $\mathbf{a}_1 = \sigma([0,0]) = [0.5, 0.5]$; $z_2 = 0.5+0.5 = 1$; $\sigma(1)\approx0.7311$.

Constraints

• Apply sigmoid after each linear step.
• Return a1 and a2 in that order.

Notes

• Caching a1 (and the pre-activations) during the forward pass is what makes the backward pass cheap — it avoids recomputing them.