Why CNNs — conv vs dense parameter count

Background

The headline reason CNNs replaced dense nets for images: weight sharing. A dense layer over a flattened image has a weight for every pixel × every output unit:

$$\text{dense} = (H\cdot W\cdot C)\cdot U + U$$

A convolution reuses one small kernel across the whole image, so its parameter count is independent of image size:

$$\text{conv} = (k\cdot k\cdot C + 1)\cdot F$$

(for F filters of size k×k over C input channels, with one bias each).

Problem statement

Implement conv_vs_dense_params(image_shape, dense_units, conv_filters, kernel_size) returning (dense_params, conv_params).

Input

• image_shape — tuple (H, W, C).
• dense_units — int, output units of the dense layer.
• conv_filters — int, number of conv filters F.
• kernel_size — int, kernel width k.

Output

Returns (dense_params, conv_params), both int.

Examples

Example 1 — the lesson's 224×224×3 case

Input:  image_shape = (224, 224, 3), dense_units = 1000, conv_filters = 64, kernel_size = 3
Output: (150529000, 1792)

Explanation: dense $= 224\cdot224\cdot3\cdot1000 + 1000 = 150{,}529{,}000$; conv $= (3\cdot3\cdot3 + 1)\cdot64 = 1792$.

Constraints

• dense = H*W*C*dense_units + dense_units.
• conv = (k*k*C + 1) * conv_filters.
• Return both as int.

Notes

• The conv count doesn't depend on H or W at all — the same kernel slides everywhere. That's how CNNs look at megapixel images with only thousands of parameters per layer.