Post-Training int8 Quantization

Background

Symmetric per-tensor int8 quantisation is the workhorse of on-device inference: 4× storage and bandwidth savings vs float32 with minimal accuracy loss. Each tensor gets one scale; values are scaled, rounded to int8, and stored. Dequantisation multiplies by the scale.

Problem statement

Implement two functions:

• quantize_int8(x) — return (q, scale) where q is an int8 array and scale = max(|x|) / 127.
• dequantize(q, scale) — return q.astype(float) * scale.

The round-trip error of any element is bounded by the scale.

Input

• x — np.ndarray of floats (any shape).
• q — np.ndarray of int8, output of quantize_int8.
• scale — float, output of quantize_int8.

Output

• quantize_int8 returns (q: np.ndarray of int8, scale: float).
• dequantize returns np.ndarray of float, same shape as q.

Examples

Example 1 — round-trip error bounded by scale

Input:  x = np.random.randn(100)
After:  q, scale = quantize_int8(x); x_hat = dequantize(q, scale)
        max(abs(x - x_hat)) < scale

Example 2 — zero input → all-zero output, scale set to 1.0

Input:  x = np.zeros(5)
Output: q = [0,0,0,0,0] (int8), scale = 1.0

Constraints

• Output q is dtype = np.int8, range $[-127, 127]$.
• scale is a plain Python float.
• All-zero input: return scale = 1.0 to avoid division-by-zero downstream.

Notes

• Symmetric vs asymmetric. Symmetric maps zero to zero; asymmetric ("zero point") shifts the scale to use the full range. Symmetric is simpler and hardware-friendly.
• Per-channel. For weights, per-channel quantisation (one scale per row/column) cuts the error further at modest extra cost.