AI 101 — What are reasoning tokens?

Reasoning tokens are the hidden text a model generates to think through a problem before giving you its answer — invisible, billed to your account, and the reason modern AI can suddenly solve hard questions.

When you ask a regular chatbot a question, it reads your words and starts writing a reply immediately. When you ask a reasoning model — OpenAI's o3, DeepSeek's R1, or Google's Gemini with "thinking" enabled — something different happens first: the model spends time generating an internal monologue. It breaks the problem down, tries approaches, catches its own mistakes, and only then speaks to you. Those hidden words are reasoning tokens.

Why it matters right now

Reasoning tokens are everywhere in 2026, even if you have never heard the term. Every major API provider now charges separately for them: OpenAI bills reasoning tokens at a different (often higher) rate than regular output tokens. When you use a coding agent like Cursor or GitHub Copilot in "think" mode, or when Anthropic's Claude works through a multi-step analysis, the invisible thinking that happens behind the scenes is made of reasoning tokens. They are the single biggest reason AI got dramatically better at math, code, and logic in the past two years — and the single biggest reason API bills went up for hard problems.

Understanding them matters because they sit at the intersection of two things readers care about: how AI actually reasons, and what you are paying for. If you have ever looked at a usage dashboard and wondered why a "simple" question cost ten times more than usual, reasoning tokens are probably why.

The mental model

Think of a reasoning model like a student allowed to think on scratch paper before writing the final answer on the exam sheet. The scratch paper is the reasoning tokens — the model's visible work. You, the teacher, only see the final answer on the sheet. But the scratch paper existed, it took time, and (in the AI version) it costs money.

Here is the key insight: reasoning tokens are not a different kind of token. They are the same tokens the model always generates — the same word-prediction machinery that produces your visible reply also produces the invisible thinking. The difference is architectural: reasoning models are trained to produce a hidden block of text first (sometimes called a "thinking block" or "reasoning trace"), and then a final answer after. The thinking block can range from a single sentence to thousands of tokens, depending on how hard the problem is.

In practice, this means reasoning tokens scale with difficulty. Ask a reasoning model "What is 2 + 2?" and it might burn two tokens thinking. Ask it to prove a theorem, debug a complex codebase, or plan a multi-step business strategy, and the thinking trace can stretch to thousands of tokens — all invisible, all billed.

An everyday analogy

Imagine a chef who, before plating your meal, first talks through the entire recipe out loud: "OK, I need to sear the salmon skin-side down for four minutes, check that the pan is hot enough, oh wait I should have started the rice first..." That narration is the reasoning tokens. You never hear it — it happens in the kitchen, not at your table. But it is what makes the dish good. And if the chef were billing you by the word, that off-menu monologue is where your bill spikes.

The crucial detail: a basic model skips the monologue entirely. It just plates. That works fine for simple orders (salad, toast), but for complex dishes (a twelve-course tasting menu), the chef who thinks aloud produces a better result — at the cost of more words and more time.

Common misconceptions

"Reasoning tokens are just chain of thought." Close, but not quite. Chain of thought is a technique — you prompt the model to show its work, and it does. Reasoning tokens are an implementation: the model was trained to produce a hidden thinking trace automatically, without you asking. Think of chain of thought as "ask the student to show their work," and reasoning tokens as "the student has been trained to always show their work — but then the teacher hides it from you." We covered what is chain of thought in an earlier explainer.

"I'm paying for extra tokens for no reason." Not exactly. Reasoning tokens demonstrably improve accuracy on hard problems — the o1 launch showed accuracy on the AIME math competition jump from 13 percent (GPT-4o) to 83 percent (o1) when reasoning was enabled. The cost is real, but so is the capability. For easy questions, most reasoning models skip or minimize the thinking block automatically.

"Reasoning tokens are always hidden." By default, yes — OpenAI and Anthropic hide the thinking from the user. But DeepSeek's R1 showed its full reasoning trace to users by default, and some tools let you toggle visibility. The trend is toward hiding: labs worry that showing the thinking leaks training signals, allows prompt injection through the reasoning trace, and gives competitors a shortcut to copying the model's abilities.

"Longer thinking always means better answers." Not necessarily. Models can think in circles, reason confidently toward wrong conclusions, or spend hundreds of tokens re-examining an already-solved step. Research from 2025 showed that beyond a certain thinking length, accuracy plateaus or even drops. More thinking is a tool, not a guarantee.

The cost equation

Reasoning tokens directly affect what you pay. On OpenAI's API, o3-mini reasoning tokens are billed at a higher per-token rate than standard output tokens. A 500-token reasoning trace on a complex coding problem might cost more than the final answer itself. For consumer-facing products, this translates to subscription pricing tiers, usage limits, and the occasional surprise bill. As one recent analysis put it, reasoning tokens have become a "model-specific API contract" — the terms under which a model agrees to think on your behalf.

For builders, this creates a real design decision: enable reasoning for accuracy, or disable it for speed and cost. Most products now offer a toggle, and the best ones auto-select based on question difficulty.

Where to learn more

OpenAI's reasoning guide explains how their models use reasoning tokens and how the billing works. The original o1 economics post breaks down when extra thinking actually pays off. And our own What is chain of thought? covers the technique that reasoning tokens automate.

When a model bills you for thinking you cannot see, should you be able to read the scratch paper? Tell us in the comments.

Sources: OpenAI — Reasoning guide · OpenAI — Economics of o1 · The Price of Thinking: Reasoning Effort as a Model-Specific API Contract · AI Midday — What is chain of thought? · AI Midday — What is a context window?