Backprop — the weight gradient is an outer product

Background

Once you have a layer's delta, its weight gradient is an outer product: the activation that fed the layer times the error that arrived at it. For $\mathbf{z} = W^\top \mathbf{a}_{\text{prev}} + \mathbf{b}$ with $W$ shaped (n_in, n_out),

$$\frac{\partial L}{\partial W} = \mathbf{a}_{\text{prev}}\,\boldsymbol{\delta}^{\top}, \qquad \frac{\partial L}{\partial \mathbf{b}} = \boldsymbol{\delta}$$

so $\partial L/\partial W$ comes out the same shape as $W$ — the sanity check to run every time.

Problem statement

Implement weight_gradient(a_prev, delta) returning the outer product $\mathbf{a}_{\text{prev}}\,\boldsymbol{\delta}^{\top}$.

Input

• a_prev — array-like activation feeding the layer, length n_in.
• delta — array-like error signal at the layer, length n_out.

Output

Returns an np.ndarray of shape (n_in, n_out).

Examples

Example 1

Input:  a_prev = [1, 2], delta = [0.5]
Output: [[0.5], [1.0]]

Explanation: outer product of a $2$-vector and a $1$-vector → shape (2, 1).

Example 2

Input:  a_prev = [1, 0, 2], delta = [1, -1]
Output: [[ 1, -1], [ 0,  0], [ 2, -2]]

Constraints

• Return np.outer(a_prev, delta) (shape (n_in, n_out)).

Notes

• Each entry $\partial L/\partial W_{jk} = (a_{\text{prev}})_j\,\delta_k$ — how much input $j$ fed neuron $k$, times the error arriving at neuron $k$.