Attention Is All You Need
Google researchers introduce the Transformer, an architecture for machine translation.
Every model on this page is a descendant. It parallelises well, so it scales with compute.
Autocomplete
2017 – 2020
Models that predict the next word. Impressive party tricks, no idea what you actually wanted.
GPT-1
OpenAI pre-trains a 117M-parameter Transformer on books, then fine-tunes it per task.
The recipe that stuck: learn language first, specialise later.
BERT
Google's bidirectional encoder sweeps the language-understanding benchmarks and ends up inside Search.
Proof that pre-trained Transformers were useful for real products, not just papers.
GPT-2
1.5B parameters, released in stages because OpenAI called it too dangerous to publish at once.
It wrote convincing paragraphs that fell apart a page later. The first public argument about release safety.
Scaling laws
Kaplan et al. show loss falls predictably as you add parameters, data and compute.
Turned "make it bigger" from a hunch into an investment plan.
GPT-3
175B parameters. Show it a few examples in the prompt and it picks up the task without retraining.
In-context learning: the model became programmable with plain text.
Chat
2021 – 2023
Models tuned to follow instructions and hold a conversation. Suddenly everyone could use one.
GitHub Copilot
Codex, a GPT-3 fine-tuned on code, ships as autocomplete inside the editor.
The first AI tool developers used every day. Code turned out to be the killer domain.
Chain-of-thought and InstructGPT
Asking a model to "think step by step" boosts its maths. RLHF makes a small model preferred over GPT-3.
Two levers beyond size: how the model reasons and what it is trained to want.
ReAct
A prompting pattern that interleaves reasoning with tool calls: think, act, observe, repeat.
The skeleton of every agent loop that followed.
ChatGPT
A free chat box on top of GPT-3.5. A million users in five days.
The moment the rest of the world noticed.
LLaMA
Meta releases weights to researchers. They leak within a week and open models take off.
Anyone with a decent GPU could now experiment, not just three labs.
GPT-4 and Claude
Same day: GPT-4 passes the bar exam in the top tenth, and Anthropic opens up Claude.
The capability jump that made people take "agents" seriously.
AutoGPT
GPT-4 in a loop with a goal and a to-do list. Top of GitHub, mostly going in circles.
The idea was right, the models were not ready. A preview, not a product.
Reasoning
2024 – early 2025
Models trained to think before answering, and given hands: a mouse, a keyboard, a terminal.
Claude 3 and Devin
Claude 3 Opus edges past GPT-4. Cognition demos Devin, billed as the first AI software engineer.
Frontier was now a race, and "AI engineer" became a pitch.
Claude 3.5 Sonnet
A mid-size model that writes better code than the previous flagships.
The model that made Cursor-style coding mainstream.
o1
OpenAI trains a model with reinforcement learning to produce long hidden reasoning before it answers.
A new scaling axis: spend more compute at answer time, get better answers.
Computer use
Claude looks at screenshots and drives the mouse and keyboard. Slow and clumsy, but it works.
Any software with a screen became a possible tool.
Model Context Protocol
Anthropic publishes an open standard for connecting models to tools and data.
Plumbing, not magic, and that is why everyone adopted it.
o3 on ARC-AGI
A reasoning model scores 75–88% on a puzzle benchmark built to resist language models.
The "they only memorise" argument lost a lot of ground.
DeepSeek-R1
An open-weights reasoning model from China, near o1 level, reportedly trained on a fraction of the budget.
Reasoning was not a moat. Nvidia lost roughly $600B in market value in one day.
Agents
2025 – now
Models that work for hours on their own: they plan, use tools, check their work and hand back results.
Deep Research and Claude Code
Agents that browse for half an hour and write a cited report. Claude Code puts an agent in the terminal with your repo.
Agents stopped being demos and became tools people paid for.
The time-horizon chart
METR measures how long a task (in human time) agents can finish reliably: it has doubled roughly every seven months since 2019.
The closest thing this road has to a speedometer.
Claude 4 and Codex
Opus 4 works on code autonomously for hours. OpenAI's Codex runs parallel coding agents in the cloud.
"Assign a ticket to the AI" became a normal workflow.
Gold at the Maths Olympiad
Experimental models from OpenAI and Google DeepMind solve five of six IMO problems in natural language.
Real novel reasoning under exam conditions, no formal-proof crutches.
GPT-5
OpenAI merges the chat and reasoning lines into one model that decides how long to think.
Reasoning became the default, not a separate product.
Gemini 3 and Claude Opus 4.5
A week apart, both labs push the frontier on coding and long agentic tasks.
The gap between the labs shrank to weeks.
This page
The first version of this site, from DNS records to deploy script, was built by an AI agent in one terminal session.
A small data point. The human still picked the idea and the domain.