How Search Engines Learned To Read Intent Before You Finished Typing
A student needed to find files buried across the early internet in 1990 and built Archie, a search tool so literal it demanded the exact file name or nothing else. Yahoo hired humans to sort the web by hand until the web stopped cooperating. Google let links vote instead.
Now AI answers before the search bar even blinks.
Read the slides as text
Slide 1
01 / 08
35 YEARS OF SEARCH INFRASTRUCTURE
How Search Engines
Learned To Read Intent
Before You Finished
Typing
1990 Archie
1994 Yahoo
1998 Google
2026 AI Search
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
From a file index built by one student in 1990 to
AI systems that answer before you click a single
link. The architecture decisions behind every
major shift.
SEARCH INFRASTRUCTURE SERIES
Slide 2
02 / 08
1990 / THE ORIGIN
Before Google, Search Meant
Knowing The Exact File Name
archie>
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
1990
Built by Alan Emtage,
a student at McGill
University
50%
Of all Montreal
internet traffic at
Archie's peak
0
Files with content
indexing. Titles only,
nothing inside
Archie indexed file names on public FTP servers so people no
longer had to hunt server by server for a download. It could not
read language or file contents, so users still needed the exact
title. The real invention was not the interface. It was the idea of
a centralized, continuously updated index, the concept every
search engine since has been built on.
SEARCH INFRASTRUCTURE SERIES
Slide 3
03 / 08
1991 TO 1994 / THE FRAGMENTATION
A Dozen Tools Raced To Fix
What Archie Could Not Do
1991
Veronica
1993
W3Catalog
1993
Architext
1994
WebCrawler
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
No single system could yet crawl, understand, and rank
the open web. Every team attacked a different piece of
the problem.
Keyword search
across Gopher
menu titles
First primitive
search built for the
web itself
Statistical word
relationships, later
renamed Excite
First engine to index
full page text, not
just titles
Every one of these systems tested a different lever: keyword
matching, statistical relevance, full text indexing. None of them
solved the trust problem. Nothing yet told a user which of a
thousand matching pages actually deserved to rank first.
SEARCH INFRASTRUCTURE SERIES
Slide 4
04 / 08
1994 / THE DIRECTORY ERA
Yahoo Proved Humans Could
Organize The Web, Until They
Could Not
YAHOO!
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
1994
Founded by Jerry
Yang and David Filo
at Stanford
100%
Human edited. Every
listing categorized by
a real person
1995
Year rival directories
and early crawlers
began to outpace
manual curation
Yahoo started as a directory, not a search engine. Editors
sorted the web into categories the way a librarian sorts a card
catalog. It worked while the web was small enough for people
to keep up with it. The moment content growth outran human
review speed, the entire model hit a ceiling that no amount of
hiring could fix.
SEARCH INFRASTRUCTURE SERIES
Slide 5
05 / 08
1998 / THE ALGORITHM SHIFT
PageRank Turned Links Into A
Trust Signal
Google
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
1998
Founded by Larry
Page and Sergey Brin
1 Link
Treated as one vote
of confidence,
weighted by the
linking page's own
authority
0
Human editors
required to rank a
page
Instead of asking an editor to judge a page, PageRank asked
the web itself. A link from a page became a vote, and votes
from already trusted pages counted more. That single
mechanism replaced manual curation with an automated
relevance signal and became the architecture blueprint search
would run on for the next two decades.
SEARCH INFRASTRUCTURE SERIES
Slide 6
06 / 08
2026 / THE CURRENT STATE
Google Now Processes More
Than 5 Trillion Searches A Year
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
90.46%
Global all device
market share,
StatCounter, May
2026
5T+
Searches processed
annually, per Google's
own disclosure
$198B
Search revenue
generated in 2025
68%
Of US Google
searches now end
without a single click
400B
Documents held in
Google's search
index
1.5pp
Year over year share
drop, the largest
single year erosion
since 2009
Google's scale has never been larger, yet its trajectory has
quietly bent for the first time in over a decade. The dominance
is real. The certainty of that dominance is not.
SEARCH INFRASTRUCTURE SERIES
Slide 7
07 / 08
2023 TO 2026 / THE AI SHIFT
Search Stopped Returning
Links And Started Returning
Answers
AI SEARCH
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
900M
ChatGPT weekly
active users reported
in early 2026
~48%
Of tracked Google
queries now show an
AI Overview
5x
Year over year
growth in AI search
referral traffic
A ranked list of ten blue links used to be the entire product.
Now the answer is synthesized on the spot, and organic click
through rates fall by more than half when an AI summary
appears above the results. The competition shifted from
ranking highest to being the source an AI model trusts enough
to cite.
SEARCH INFRASTRUCTURE SERIES
Slide 8
08 / 08
THE PATTERN
Every Era Rewarded The Same
Skill: Extracting Signal From
Noise
1990
Archie's signal was the file name, indexed by machine for the
first time
1994
Yahoo's signal was human judgment, until volume made it
impossible to scale
1998
Google's signal was the link graph, letting the web rank itself
2026
AI search's signal is earned citation, trusted sources over
ranked pages
NighT OwL
ENGINEERING BEHIND BIG SYSTEMS
The interface changed four times in thirty five years. The
underlying job never did: find the trustworthy signal faster than
anyone else can fake it. Whoever builds that layer next owns
the following decade of search.
SEARCH INFRASTRUCTURE SERIES