AI 101 — What is an AI hallucination?

An AI hallucination is a statement a chatbot or model produces with total confidence that simply isn't true. The model isn't lying, glitching, or behaving maliciously — it is doing exactly what it was trained to do, which is to generate the next most plausible string of words. Sometimes the most plausible thing is wrong, and the result is a confident falsehood you can't always spot without checking.

Familiarity should not breed contempt for the term: hallucinations are still the single most important reason you cannot trust an AI answer at face value. They keep surfacing in this publication's coverage — as the stated caveat in a medical-triage study, the reason an open-source agent misreads your documents, the thing courts now sanction lawyers for. Even as models have grown vastly more capable, hallucination has not gone away, and the researchers who study it argue the problem is partly baked into how models are trained and scored.

The mental model

A large language model is an advanced next-word predictor. During pretraining it absorbed enormous amounts of text and learned patterns — which words tend to follow which words — rather than a database of verified facts. Critically, it saw no labels marking statements true or false; it only saw examples of fluent language and approximated the underlying pattern. That means the model can't distinguish "plausible" from "real." For facts that follow consistent rules — spelling, grammar, parentheses — patterns make errors vanishingly rare. But arbitrary low-frequency facts, like a person's birthday or the exact citation of a court case, can't be predicted from patterns alone. When the model must answer one, it produces whatever fits the pattern best. In interviews, the researchers who wrote OpenAI's paper on the topic asked a widely used chatbot for the title of a colleague's PhD dissertation; it volunteered three different confident answers, none of them right.

Worse, the incentives of how models are graded push them toward guessing rather than honesty. Most tests score models only on accuracy — the share of questions answered correctly. On a multiple-choice test where a blank guarantees zero but a guess might get lucky, the guesser out-scores the cautious student. It's the same for models: asked a birthday it doesn't know, a model that guesses "September 10" has about a one-in-365 shot at credit, while a model that says "I don't know" earns nothing. Over thousands of questions, the guesser looks better on the leaderboard. That's why, as OpenAI's team put it, hallucination persists: the training and evaluation process rewards lucky guesses over admitting uncertainty.

The barstool-analogy friend

Here's the everyday version: imagine a trivia-night friend who has absorbed an enormous amount of trivia but refuses to say "I don't know." Pressed on the capital of a country they haven't studied, they don't say "no idea" — they confidently name whichever capital feels most plausible, delivered with the same conviction as a fact they actually know. Most of their answers are right. But the wrong ones are indistinguishable in tone from the right ones, which is exactly the danger. Now imagine a million such friends answering about a million topics, at machine speed, and you understand both the power and the hazard of a modern model.

Common misconceptions

"A hallucination means the model is broken (or sentient)." Neither. It's an expected side effect of a system that predicts plausible language and lacks a built-in fact-checker. Calling it a glitch misreads what the technology is doing.

"New models simply won't hallucinate." Bigger, reasoning-focused models reduce the rate, but don't eliminate the incentive problem. In one OpenAI comparison on a factual benchmark, an old model answered correctly 24 percent of the time (and guessed wrong 75 percent of the time), while a newer one was right only 22 percent of the time but abstained on half the questions and erred far less — a better trade for real use, yet still not zero.

"If I use my own documents, the model can't make things up." Grounding a model in your own material — the technique behind RAG — dramatically cuts hallucination, but it doesn't remove the possibility. The flaky facts merely shift toward confident misreadings of your sources, so verification still matters.

"Hallucination is a niche research topic." It has real, expensive consequences. In the legal world alone, courts have sanctioned lawyers for filing briefs that cite cases that don't exist — commitments to accuracy are being turned into legal liability in August 2026.

Where to learn more

For the deepest explanation, read the paper that reframed the subject: OpenAI's researchers, with Georgia Tech's Santosh Vempala, argue that hallucinations flow from next-token training and from scoreboards that reward guessing — and propose penalizing confident errors more heavily than uncertainty. In practice, the tools that reduce hallucinations overlap with the techniques behind what is RAG, which anchors answers in real sources, while letting a model reason before answering — the territory of what is chain of thought and the reasoning tokens that power it — are what the newer, more abstemious models lean on. None are perfect; all beat trusting bare autocomplete.

Related reading: What is RAG? · What is chain of thought? · What are reasoning tokens?

When was the last time you caught a chatbot making something up — and did you check before you believed it? Tell us in the comments.

Sources: OpenAI — Why language models hallucinate · Kalai & Vempala et al., Why Language Models Hallucinate (arXiv) · Reuters — Connecticut's top court sanctions lawyer over AI-fabricated citations · Scientific American — Why lawyers keep citing fake cases invented by AI