#242The residual Jacobian I + ∂f/∂xEasyTransformersCNNCalculusLLMs
The residual Jacobian I + ∂f/∂x
Background
A residual block computes y = x + f(x), so its Jacobian gains an identity term:
That I is the gradient highway: even if ∂f/∂x collapses to zero, the gradient through the skip is still 1, so it can't vanish.
Problem statement
Implement residual_jacobian(df_dx) returning I + df_dx.
Input
df_dx— the sublayer Jacobian∂f/∂x, a square array(n, n).
Output
Returns an np.ndarray (n, n): the identity matrix plus df_dx.
Examples
Example 1 — vanished sublayer gradient still gives identity
Input: df_dx = [[0, 0], [0, 0]]
Output: [[1, 0], [0, 1]]
Example 2
Input: df_dx = [[1, 0], [0, 1]]
Output: [[2, 0], [0, 2]]
Constraints
- Return
eye(n) + df_dx. - Shape
(n, n).
Notes
- The whole point: a zero sublayer Jacobian leaves the full Jacobian at
I, never0— the gradient survives any depth.
Python
Loading...
▶ Run executes the 3 visible sample tests below in your browser. Submit runs the full suite — including hidden tests — on the server for an official verdict.
- •Example: zero sublayer gradient -> identity
- •Reference: identity sublayer -> 2I
- •Sample: 2x2 dense off-diagonal