SimpleRNN forward — outputs over a sequence

Background

The full RNN forward pass threads a hidden state through the sequence and emits an output at each step (the lesson's SimpleRNN.forward):

$$\mathbf{h}_t = \tanh(\mathbf{x}_t W_{xh} + \mathbf{h}_{t-1} W_{hh} + \mathbf{b}_h), \qquad \hat{\mathbf{y}}_t = \mathbf{h}_t W_{hy} + \mathbf{b}_y$$

starting from $\mathbf{h}_0 = \mathbf{0}$, with the same weights reused at every step.

Problem statement

Implement rnn_forward(X, W_xh, W_hh, W_hy, b_h, b_y) returning the stacked outputs.

Input

• X — shape (T, D): input at each timestep.
• W_xh — (D, H), W_hh — (H, H), W_hy — (H, O).
• b_h — (H,), b_y — (O,).

Output

Returns an np.ndarray of shape (T, O): the output $\hat{\mathbf{y}}_t$ at each timestep.

Examples

Example 1 — single timestep

Input:  X = [[1]], W_xh = [[1]], W_hh = [[0]], W_hy = [[1]], b_h = [0], b_y = [0]
Output: [[0.76159416]]

Explanation: $h_0 = \tanh(1)$; $\hat{y}_0 = h_0 \approx 0.7616$.

Constraints

• Start from h = zeros(H) where H = W_hh.shape[0].
• Each step: h = tanh(x_t @ W_xh + h @ W_hh + b_h), then y = h @ W_hy + b_y.
• Reuse the same weights every step; return shape (T, O).

Notes

• Keep only outputs[-1] for many-to-one; keep all for many-to-many. Differentiating through this loop is backpropagation through time.