Magnitude Pruning

Background

Magnitude pruning is the simplest model-compression baseline: zero out the smallest-magnitude weights. With sparsity $s$, $s\%$ of the weights become 0, and inference engines that support sparse matmul speed up proportionally.

Problem statement

Implement magnitude_prune(weights, sparsity). Find the threshold = the sparsity-quantile of |weights|. Build a boolean mask where the weight survives iff its magnitude is strictly above the threshold. Return (pruned_weights, mask).

Input

• weights — np.ndarray of floats (any shape).
• sparsity — float in $[0, 1)$, the fraction to zero out.

Output

Returns (pruned: np.ndarray, mask: np.ndarray of bool). Both have the same shape as weights.

Examples

Example 1 — half pruned

Input:  weights = [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4], sparsity = 0.5
Output: mask = [True, True, True, ...] for the 5 largest in absolute value
        pruned has zeros at the 5 smallest-magnitude positions

Example 2 — no pruning

Input:  sparsity = 0.0
Output: pruned == weights, mask all True

Constraints

• 0.0 <= sparsity < 1.0. ValueError outside.
• Mask is True where the weight survives, False where pruned to zero.
• pruned = weights * mask (elementwise).
• Original weights array is not mutated (return a copy).

Notes

• What it doesn't capture. Two weights of similar magnitude may matter very differently — magnitude is a noisy proxy for importance. Movement pruning uses gradient information; Lottery Ticket Hypothesis prunes after training and retrains.
• Structured vs unstructured. This is unstructured (any individual weight may be pruned). Structured pruning removes whole channels / heads — easier to actually accelerate on real hardware.