RNN output — project the hidden state

Background

After updating the hidden state, an RNN reads out a prediction with a linear output projection:

$$\mathbf{y}_t = W_{hy}\,\mathbf{h}_t + \mathbf{b}_y$$

(For classification you'd add a softmax; here it's the raw projection.) Following the lesson's code convention, with h_t a row vector this is h_t @ W_hy + b_y where W_hy has shape (H, O).

Problem statement

Implement output_projection(h, W_hy, b_y) returning $\mathbf{h}\,W_{hy} + \mathbf{b}_y$.

Input

• h — hidden state, length H.
• W_hy — shape (H, O).
• b_y — length O.

Output

Returns an np.ndarray of length O: the output vector.

Examples

Example 1 — identity readout

Input:  h = [1, 2], W_hy = [[1, 0], [0, 1]], b_y = [0, 0]
Output: [1 2]

Example 2 — with bias

Input:  h = [1], W_hy = [[2, 3]], b_y = [1, 1]
Output: [3 4]

Explanation: $[2\cdot1, 3\cdot1] + [1, 1] = [3, 4]$.

Constraints

• Compute h @ W_hy + b_y.
• Return shape (O,).

Notes

• This is the only place the output dimension O enters; the hidden size H is decoupled from how many things you predict.