AI 101 — What is chain of thought?

Chain of thought is the step-by-step reasoning a language model generates before it gives a final answer — the "show your work" part of AI problem-solving. Instead of jumping straight from question to answer, the model writes out its intermediate steps, and that simple change measurably improves how well it handles math, logic, and anything with multiple stages.

The idea surfaced in a January 2022 paper from Google Research: the authors found that asking a model to work through a problem aloud — by showing it a few solved examples with their reasoning included — lifted a 540-billion-parameter model's score on the GSM8K grade-school math benchmark from 17.9 percent to 56.9 percent, with just eight examples as prompts. No retraining, no new architecture; the model already knew how to reason, it just needed to be asked to show its work.

Why it's in the news

Chain of thought is the quiet engine behind this week's biggest stories. Our deep dive on Monday covered how Anthropic, OpenAI, and Google hide their models' step-by-step reasoning in encrypted blocks — and how researchers decoded those traces from Claude, GPT, and Gemini models by feeding them to a weaker sibling model. The reason the labs bother hiding the thinking at all is that a model's chain of thought is its most valuable and most sensitive output: it reveals how the model was trained, it can leak private data, and it hands competitors a shortcut to copying the model's abilities.

It also explains a phrase you now see on every API pricing page: "reasoning tokens." When OpenAI launched its o1 models in September 2024, it introduced a new class of model that thinks before it answers — and bills you for the thinking. DeepSeek's R1 models made the idea famous in January 2025 by showing their full chains of thought to users. Since then, "reasoning models" have become the default for hard tasks, and the tokens they burn while thinking have become a line item in every AI budget.

The mental model

A language model is, at heart, a machine for predicting the next word. Chain of thought works by changing what the model predicts: instead of answering immediately, it predicts a sequence of reasoning tokens — an internal monologue like "the area of a circle is πr², so first I need the radius" — and only produces its answer after that monologue reaches a conclusion. The steps are just more tokens, generated by the same machinery; the magic is that forcing the model to commit to intermediate conclusions keeps it from collapsing the whole problem into one lucky (or unlucky) guess. For a hard problem, getting the answer right in one shot is like hitting a dartboard blindfolded; working through the steps is like walking up to the board first.

An everyday analogy

Imagine a student taking a math exam. One student circles answers instantly; the other writes out every step — "I factored the denominator first, then simplified." The second student gets more questions right, and when they do slip up, the work shows exactly where. That's chain of thought: it's showing your work on an exam, except the exam is a math problem, a code bug, or a travel itinerary, and the student is a model that has read most of the internet. The teacher (you) gets to see the reasoning — or, increasingly, only the final answer, when the lab running the model decides to keep the work private.

Common misconceptions

"Chain of thought is the model's actual thinking." Not exactly. It's text the model generates because it was trained or prompted to — a useful proxy for reasoning, not a faithful transcript of whatever the model "really" does. Researchers are still debating how closely written chains of thought match the underlying computation.

"It's just for math problems." Arithmetic was the first demonstration, but chain of thought works wherever problems have steps: coding, planning, debugging, multi-hop questions. The 2022 paper showed gains across arithmetic, commonsense, and symbolic reasoning.

"More thinking always means better answers." Not automatically. Longer chains cost more tokens and take longer, and models can "think" in circles or reason confidently toward a wrong answer. The benefit is real but has limits — which is exactly why labs study how much thinking is worth paying for.

"It's a prompting trick." It started as one, but reasoning models like o1 and R1 are trained to produce chains of thought internally — the prompting-era trick became a core model capability. The frontier has moved from coaxing reasoning out to deciding whether to let users see it.

Where to learn more

The original 2022 paper is surprisingly readable and contains the now-famous benchmark numbers. OpenAI's reasoning guide explains how reasoning models work and how the tokens are billed. DeepSeek's R1 release notes are the best look at an open-weight model that shows its thinking in public — the opposite of the encrypted-block approach. And our own analysis of the encrypted chain-of-thought paper walks through what happens when the "secret" reasoning is pried open.

When a model shows you its work, are you watching real reasoning — or a performance? Tell us in the comments.

Sources: Wei et al. — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models · AI Midday — Cracking the encrypted chain-of-thought in frontier LLM APIs · OpenAI — Reasoning guide · OpenAI — Evaluating chain-of-thought monitorability · DeepSeek — DeepSeek-R1 release notes