AI 101 — What is a diffusion language model?

A diffusion language model writes text the way an image generator paints a picture: it starts with a block of noise and refines the whole thing over several passes, instead of typing one word at a time left to right. Almost every AI you have used does the typing. This is the main rival design, and in the last week it went from research curiosity to something running real phone calls and coding agents.

The reason it is in the news is Inception's Mercury 2.5 release, which the company calls the most capable diffusion language model on the market and, by its own account, the largest ever trained. Google shipped an open experimental one called DiffusionGemma in June. The pitch is speed: if you do not have to wait for word 41 before starting word 42, you can finish a paragraph in a fraction of the time.

The one-paragraph mental model

A normal large language model is autoregressive — each token (a chunk of a word, not a whole word) is predicted from the tokens before it, and the model cannot start the next one until the previous one exists. That is why text streams out word by word. A diffusion language model takes a different approach borrowed from image generation: fill a fixed-length block of, say, 256 positions with meaningless placeholder tokens, then repeatedly ask the model to guess what should be there. Each pass replaces the guesses it is least sure about and keeps the confident ones, so the block sharpens over a handful of steps. Because every position in the block is being considered at once, the model can see both the beginning and the end of what it is writing — it is genuinely bidirectional, where an autoregressive model is locked into looking backwards only. Googling "masked diffusion" or "discrete diffusion" will get you the technical version; the key point is that noise-to-text replaces token-by-token.

The analogy: a typewriter versus a printing press

Google's own team uses this one, and it is the right one. An autoregressive model is a typewriter: one key at a time, in order, and the machine is mostly idle waiting for the next keystroke. On a shared cloud server that idleness does not matter, because the GPU is juggling thousands of other people's requests in the gaps. On your own machine, serving one user, it means an expensive chip sitting around doing nothing.

A diffusion model is a printing press: it lays out the whole page at once and stamps it. More work per pass means the hardware is actually busy, which is why Google says DiffusionGemma reaches up to four times faster generation on a dedicated GPU, and why it also warns the advantage shrinks in high-traffic cloud serving where autoregressive models already batch efficiently. The real-world numbers from Inception's users land the same way: a voice-agent company reports median response latency near 170 milliseconds, and a coding-tool company says moving long-session summarization onto Mercury cut that step from roughly 150 seconds to 27 seconds.

What people get wrong

"It thinks in parallel, so it is smarter." No. Speed and quality are separate axes. Diffusion models are competitive at their size, and a Renmin University–Ant Group team reported this year that a 30-billion-parameter mixture-of-experts diffusion model approached a comparable autoregressive model while using about two-thirds of the training tokens — but nobody has shown a diffusion model ahead of the frontier on hard reasoning.

"It revises, so it cannot make mistakes." The refinement process fixes local awkwardness, not factual errors. A diffusion model hallucinates for the same reason any model does: it is matching patterns, not checking facts.

"It will replace the transformer." Diffusion language models are transformers too. The change is in how text is generated, not what the network is made of.

"It is free speed." Parallel decoding costs more compute per output than a single sequential pass; you are trading idle hardware for busy hardware. It pays off where latency matters more than raw throughput.

Where to learn more

Start with the LLaDA paper, "Large Language Diffusion Models" — the 8-billion-parameter model that showed a diffusion design could be trained from scratch and reach results comparable to strong autoregressive models of the same size. For the middle ground, look up block diffusion, which interpolates between the two approaches by generating short blocks in order. Google's DiffusionGemma release notes are the clearest plain-English write-up, and it ships under an open license if you want to run one.

Related reading: What is a token in AI? · What is speculative decoding? · What is a transformer? · Local LLMs vs cloud APIs: what's the difference?

Would you trade a little raw capability for a response that arrives five times faster? Tell us in the comments.

Sources: Large Language Diffusion Models (LLaDA, arXiv) · Google DeepMind — DiffusionGemma · Inception — Introducing Mercury 2.5 · Block Diffusion (arXiv)