AI 101 — What is an AI benchmark?
If you read AI news for a week, you'll see the same strange nouns over and over: SWE-bench, Terminal-Bench, GPQA, FrontierMath. Every model launch ships a table of percentages. Nobody explains what the test actually is.
An AI benchmark is a fixed set of tasks with a fixed, automatic way to grade them — a standardized exam for models, where every model gets the same questions and the answers are scored by code rather than by opinion.
That last part is the whole trick. A benchmark isn't just a question bank; it's a question bank plus a grader that doesn't get tired, doesn't argue, and doesn't have a favorite lab.
Why it matters right now
Benchmarks are the scoreboard the industry runs on, and the scoreboard is under strain. Terminal-Bench, the agent benchmark hosted by Stanford's Harbor project and the Laude Institute, is now on version 4.0 — its team has spent recent releases removing "saturated" tasks, fixing broken ones, and calibrating time and memory limits so that agents stop failing for reasons unrelated to intelligence. On the widely used Terminal-Bench 3.0 release, the best model scored about 34 percent. That's the honest state of the art on hard, long-running computer tasks.
The strain shows up in two places. First, benchmarks get solved: SWE-bench launched in 2023 with 2,294 real GitHub issues drawn from 12 popular Python repositories, and the best model at the time — Claude 2 — resolved under 2 percent of them. Coding models now clear most of it, which is why SWE-bench Verified, a human-filtered 500-instance subset, became the number everyone quotes. Second, benchmarks get gamed. The Terminal-Bench team added an integrity policy after finding agents that looked up published solutions online; that now scores zero.
The mental model
A benchmark has three parts: tasks, a harness, and a grader. The tasks are the questions. The harness is the machine that hands each task to the model and steps out of the way — for a coding benchmark that usually means a sealed container with a real codebase and a real terminal. The grader checks the outcome, not the reasoning.
Verifiability is what separates a good benchmark from a vibe. SWE-bench's tasks come with hidden tests written by the human developers who fixed the original bug; a model "resolves" an issue only if the code passes those tests. Terminal-Bench uses the same philosophy in a shell: each task ships with a script that verifies whether the job actually got done — a server that answers, a binary that compiles, a file that matches. Nobody reads the model's work and assigns a grade. Either the tests pass or they don't.
That's also why benchmarks cluster where they do: code, math, and terminal work, because those have unambiguous right answers. Judging a marketing email is a different and much harder problem, and the industry mostly avoids pretending otherwise.
An analogy
Think of a driving test. The route is fixed, the maneuvers are the same for everyone, and the examiner has a clipboard with checkboxes: mirror check, signal, stop at the line. You don't get credit for a confident explanation of why you skipped the stop sign.
A benchmark is that clipboard. The route is the task set, the checkbox sheet is the grader, and the score is the share of maneuvers you completed. What it will never tell you is whether you're a good driver in rain, in a city you've never seen, at night. It measures what's checkable, and that's both its value and its limit.
Common misconceptions
"Higher score means a better model." Not across benchmarks, and often not within one. A score is measured under one harness, with one agent scaffold, at one effort setting. Change the harness and the ranking can shuffle — which is precisely why the Terminal-Bench team now publishes 95 percent confidence intervals alongside every result. A one-point gap on a 74-task benchmark is noise, not a verdict.
"It's the same as a real evaluation." A benchmark is a proxy for work you care about; an eval is the thing you care about, measured on your own tasks with your own data. The benchmark tells you which model to shortlist. It doesn't tell you it will handle your codebase.
"The model hasn't seen the questions." Sometimes it has. Contamination — test items leaking into training data — inflates scores silently, and detecting it is genuinely hard; recent work on the problem shows that a contamination audit failing to find evidence can mean "clean" or simply "the audit wasn't sensitive enough." Google DeepMind's response is a double-blind pilot that seals external test prompts in a cryptographic box so a provider can't absorb them into training, evaluated with the Singapore AI Safety Institute and others.
"Scores are stable over time." They aren't, and the task sets move underneath them. Terminal-Bench treats benchmarks as software that ships versions, retires solved tasks, and reruns leaderboards — so a 2025 number and a 2026 number on "the same" benchmark may not be comparable.
Where to learn more
Start with the two benchmarks named most often in AI news. The SWE-bench site publishes the task family, the paper, and live leaderboards you can filter by model. Terminal-Bench's docs and blog explain how a terminal task is constructed and why the team keeps rewriting them — the "Continuous Benchmarks" post is a short, unusually frank account of how these exams are maintained.
For a sense of how fragile the whole enterprise is, our own coverage of Google's double-blind evaluation pilot is the clearest explanation of contamination in plain English.
Related reading: What is an AI agent? · What is model quantization? · What is fine-tuning?
Ever picked a model because of a benchmark number and regretted it? Tell us in the comments.
Sources: SWE-bench · SWE-bench paper (arXiv) · Terminal-Bench · Terminal-Bench 4.0 release notes · Google DeepMind on double-blind evaluations · When Is Benchmark Contamination Detectable? (arXiv)
