Deep Computer Vision (CNN)
Lesson 3From pixels to perception
How raw pixel grids become meaningful visual patterns
From pixels to perception
Bridging human intuition and machine perception
The core question is technical: how do we encode a vision task humans do effortlessly, recognizing a face, telling two objects apart, reading a scene, into a machine that has no built-in intuition? It comes down to turning pixels into features.
How computers represent images
At the lowest level, images are fundamentally numerical.
| Kind | What each pixel is | Shape |
|---|---|---|
| Grayscale | A single intensity value | 2D matrix (height, width) |
| Color (RGB) | A triplet of numbers, red, green, and blue intensities | 3D tensor (height, width, 3) |
This structured representation is ideal for feeding into neural networks. Unlike text, which must be tokenized and embedded, images are already in a dense numerical format. The challenge is not converting them into numbers, but interpreting the numbers meaningfully.
Pixels → perception: features build up stage by stage
A CNN learns this hierarchy automatically: each layer combines the previous layer's features into something more meaningful.
Understanding the learning tasks: regression and classification
Once an image is represented numerically (as pixels), we can train deep learning models (neural networks) to perform specific tasks. These tasks typically fall into two broad categories:
Regression
Outputs continuous values.
Examples in vision
- Predicting the age of a person from a face image
- Estimating the depth of each pixel in a scene
- Forecasting motion vectors
Classification
Outputs discrete class labels.
Example: given an image of a historical figure, determine whether it is Abraham Lincoln, George Washington, or another U.S. president.
The output is one of K possible categories, and the model must learn to assign high probability to the correct one.
Feature extraction: the key to recognition
The running example
Let’s ground this with an example. Consider the task of identifying whether a portrait is Abraham Lincoln or George Washington. Humans perform this by relying on distinctive features:
Human cues (how we read the portrait)
| Step | Cue |
|---|---|
| 1 | Facial structure |
| 2 | Hairstyle |
| 3 | Beard |
| 4 | Historical clothing |
What the machine must do instead
For a machine to do this, it must also extract features, but unlike humans, it needs these features to be explicitly defined (traditional neural-networks) or automatically learned (whooo, this sounds interesting..). This brings us to the heart of deep learning for vision: representation learning.
Where do features come from? (before the CNN “magic”)
| Approach | What you actually do |
|---|---|
Classical ML | In classical machine learning, we have seen how image processing techniques have helped us extract features that help us understand some nuances about the images, not using raw pixels as input features for neural networks. |
Traditional neural network, option A | Feed it raw pixels of a 28×28 image as 784 input features. |
Traditional neural network, option B | Use classical feature engineering to extract specific and precise features, acting as a domain expert or subject matter expert for computer vision problems. Concatenate those features into a vector and feed that input feature vector to the neural network. |
Check your understanding
1 / 6An RGB image of size 224 × 224 has how many raw pixel values (channels × height × width)?
Practice it yourself
Turn pixels into the numbers an MLP eats — and see why flattening is a blunt instrument. Real Python, hidden tests, no setup.