AI 101 — What is a context window?

A context window is the amount of text a language model can look at at one time — its working memory. It is measured in tokens (roughly three-quarters of an English word each), and it caps how much of your prompt, your conversation, or your document the model can actually see when it writes a reply. Everything you want the model to consider must fit inside that window, or it gets cut off, summarized, or ignored.

Why it's in the news

Context size has quietly become one of the biggest selling points in AI — and this week is a good example. Upstage shipped Solar Pro 4 with a 512K-token context window, and DeepSeek pushed V4 Pro to general availability with a 1-million-token window. Those numbers sound like marketing trivia, but they decide what you can actually do with a model: a 200K window holds roughly 150,000 words — about 500 pages — enough for a whole codebase directory, a full legal contract, or a long technical manual. A million tokens is a bookcase. The industry moved from GPT-4's 8,000 tokens in March 2023 to a million in less than a year, and every jump has unlocked a new class of task: chat became document review, then document review became "read my entire repository and fix the bug."

The mental model

Think of the model as a worker with a finite desk. Your prompt is everything you lay on the desk: the question, the attached files, the conversation history, the instructions. The model can only work with what is physically on the desk — it reads across all of it, weighs it, and writes its answer using it. Anything that doesn't fit never gets seen. When the desk is nearly full, the worker has options: you can summarize what's there to make room (condensing earlier messages), you can shove a file off the edge (it's gone from the model's view), or you can move to a bigger desk — a model with a larger context window. The catch is that a bigger desk is more expensive to maintain and slower to search, which is why context size is a genuine engineering trade-off, not just a spec-sheet number.

An everyday analogy

Imagine a chef cooking from a stack of recipes. The recipes she can lay out on the counter at once are her context window: with a small counter, she can follow one dish at a time and must put completed recipes away before starting the next. Give her a huge counter, and she can have the appetizer, the main course, the dessert, the wine notes, and the guest list all in front of her — she can cross-reference them, keep the whole meal consistent, and remember that the guest of honor is allergic to garlic. But a kitchen with unlimited counter space still has limits: the ingredients in the pantry (the model's training knowledge) are not on the counter, and the more recipes she spreads out, the harder it is to notice she's misread one in the middle. The context window is the counter, not the pantry — and most AI misunderstandings come from confusing the two.

Common misconceptions

"A bigger context window means a smarter model." No. It means a model that can see more at once — but research has shown that models recall details best from the start and end of long inputs and often miss things buried in the middle. A huge window doesn't guarantee careful attention to everything inside it; it just makes it possible.

"The model remembers what I told it." Not after the session. The context window is working memory, not long-term memory: when a conversation ends, the window is discarded, and the model retains nothing unless it was saved into its training data. You can paste a project's history into a new session and the model will act as if it knows it — but that's the window, not a memory.

"Tokens are words." Roughly three-quarters of a word, on average — and the conversion varies wildly: "the" might be one token while a long technical term eats several. That's why context sizes in tokens (200K, 1M) translate to fewer words than they sound like.

"Long context is free now." It's cheaper than it was — early long-context models charged premium rates for the extra tokens — but every token in your prompt costs money and compute time. Filling a million-token window on every request is expensive, which is why real products summarize aggressively instead of dumping everything in.

Where to learn more

IBM has a clean primer on context windows and tokenization if you want the mechanics. Taskade's history of the context window is the best single page for the milestone numbers — when each lab hit 100K, 200K, and 1M tokens. Anthropic's original 100K announcement and Google's Gemini 1.5 launch post are the primary sources for the two biggest jumps. And this week's launches — Upstage's 512K Solar Pro 4 and DeepSeek's 1M-token V4 Pro — show where the race stands right now.

What's the longest document you'd trust a model to read in one go — and where do you draw the line? Tell us in the comments.

Sources: IBM — What is a context window? · Taskade — From 4K to 1M: The History of the Context Window · Anthropic — 100K context windows · Google — Gemini 1.5: Our next-generation model · AI Midday — Upstage ships Solar Pro 4, an agentic model with 512K context · AI Midday — DeepSeek ships V4 Pro, ending its flagship's four-month preview