Engineers once wrote rules for every edge, corner, and shadow just so a computer could guess "cat."
Years of tweaking, still fooled by bad lighting. Then a network looked at raw pixels and taught itself the concept of a cat, beating every hand written rule in one attempt.
Read the slides as text
Slide 1
15.3%
AlexNet error rate, 2012
3.57%
ResNet error rate, 2015
5.1%
Human baseline error
1,000
object categories in the
benchmark
SYSTEM TEARDOWN
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
How Computers
Learned To See
From raw pixel grids to object recognition that now beats human
accuracy. This is the architecture behind that leap, and what actually
changed along the way.
NIGHT OWL / SYSTEMS DESK
01 / 08
Slide 2
THE STARTING POINT
NighT OwL
PHASE 00
A Photo Is Just Numbers
A computer never sees a face, a street, or a cat. It sees a grid
of pixel intensity values, three numbers per pixel for red,
green, and blue. Every layer of understanding after that has to
be built from scratch through math, because none of it exists
in the raw file.
3,000,00
numeric values
inside a single
1000x1000 pixel
color photo
0
built in concept of
edge, shape, or
object in that raw
grid
256
possible intensity
levels per color
channel, per pixel
16.7M
possible color
combinations a
single pixel can
represent
NIGHT OWL / SYSTEMS DESK
02 / 08
Slide 3
PHASE 01 / PRE 2012
NighT OwL
THE MANUAL ERA
Decades Of Hand Built Rules
Before deep learning, engineers coded the rules themselves.
Detectors like SIFT and HOG hunted for edges, corners, and
gradients using formulas humans designed by hand, then fed
those features into a classifier. It worked, until the picture
changed even slightly.
Raw Image
pixel grid input
→
Edge Detector
hand coded gradients
→
Feature Vector
manual descriptors
→
Classifier
SVM prediction
25.8%
best top 5 error rate on
ImageNet in 2011, the final
year before deep learning
took over
Fragile
hand built features broke
under new lighting, angles, or
backgrounds outside their
design
14M+
images catalogued across
20,000+ categories in the full
ImageNet dataset, 2009
NIGHT OWL / SYSTEMS DESK
03 / 08
Slide 4
PHASE 02 / 2012
NighT OwL
THE BREAKTHROUGH
One Network Changed Everything
In 2012, an eight layer convolutional neural network called
AlexNet entered the ImageNet competition and beat the entire
field by a margin nobody had seen before. It learned its own
features directly from pixels instead of relying on hand built
rules.
Runner Up 2012
AlexNet 2012
26.2% error
15.3% error
8
layers deep, five
convolutional and
three fully
connected
60M+
trainable
parameters learned
directly from image
data
1.2M
training images
spanning 1,000
object categories
5 to 6
days spent training
AlexNet across two
GPUs running in
parallel
NIGHT OWL / SYSTEMS DESK
04 / 08
Slide 5
THE MECHANISM
NighT OwL
UNDER THE HOOD
Filters Learn What An Edge Is
A convolution layer slides small filters across the image, each
one learning to react to a pattern such as an edge or a color
shift. Pooling shrinks the result while keeping the strongest
signals. Stack enough of these and simple edges combine
into shapes, then parts, then full objects.
Pixels
input grid
→
Conv + ReLU
edges detected
→
Pooling
signal compressed
→
Conv + ReLU
shapes detected
Pooling
signal compressed
→
Fully Connected
features combined
→
Prediction
object identified
11x11
size of AlexNet's first layer
filters scanning the raw
image
Shared
weights let one filter detect
the same pattern anywhere in
frame
5 + 3
convolutional layers feeding
into fully connected layers in
AlexNet
NIGHT OWL / SYSTEMS DESK
05 / 08
Slide 6
PHASE 03 / 2012 TO 2015
NighT OwL
THREE YEARS, TOTAL REVERSAL
Error Prone To Superhuman
Once the deep learning approach proved itself, progress
compounded fast. Networks kept getting deeper, and the error rate
kept collapsing, until machines crossed the human accuracy line
entirely.
2012 AlexNet
2014 GoogLeNet
2015 ResNet
Human Baseline
15.3%
6.7%
3.57%
5.1%
152
layers deep in ResNet, made
trainable through residual
connections
2015
first year a network's error
rate fell below the human
benchmark
1,500
test images used to establish
the 5.1% human accuracy
baseline
NIGHT OWL / SYSTEMS DESK
06 / 08
Slide 7
PHASE 04 / TODAY
NighT OwL
OUT OF THE LAB
Now It Runs The Real World
The same convolution and pooling principles from the ImageNet
era now sit inside everyday infrastructure, running constantly and
mostly unnoticed.
AUTONOMOUS DRIVING
Lane, pedestrian, and obstacle detection running on
live camera feeds in real time
MEDICAL IMAGING
Scan analysis that flags tumors and anomalies faster
than manual review alone
RETAIL AND LOGISTICS
Checkout free stores and warehouse robots
identifying items on the move
CONSUMER PHOTOS
Automatic face grouping, object search, and scene
detection on every phone
NIGHT OWL / SYSTEMS DESK
07 / 08
Slide 8
THE TAKEAWAY
NighT OwL
CLOSING INSIGHT
The Architecture Stayed.
The Scale Changed.
Convolution and pooling were first described in 1989. What
actually flipped computer vision from a research curiosity to
critical infrastructure was not a new idea, it was more labeled
data, more compute, and networks willing to go deeper than
anyone had tried before.
1989
year convolutional networks
were first described for vision
tasks
2012
year enough data and
compute existed to prove the
idea at scale
22B
parameters in today's largest
vision transformers, up from
60 million in 2012
NIGHT OWL / SYSTEMS DESK
08 / 08