AI 101 — What is a world model?

Robotics companies shipped at least ten "world models" between May and August. The phrase is now everywhere in AI — and means three different things depending on who says it.

A world model is a learned internal simulation of an environment: give it what you see and what you're about to do, and it predicts what the world looks like next. That's the one-sentence version. A language model predicts the next word; a world model predicts the next state — the next camera frame, the next joint angle, the next arrangement of objects on a table. Build one and an AI system can ask "what happens if I do this?" without actually doing it.

It matters right now because the field has stopped treating it as a research curiosity. This morning Unitree claimed its UnifoLM-X2-1.0 model drove a G1 humanoid through an autonomous sparring session — no operator, no scripted moves — by predicting scene changes and an opponent's motion fractions of a second before the joints moved. That's a world model inside a control loop, which is a much harder job than generating a plausible video.

The mental model: three parts, one loop

Strip away the branding and almost every world model has the same shape. Something compresses the raw observation — a camera image, a video clip, a pile of sensor readings — into a compact state the model can reason about. Something predicts how that state evolves, usually conditioned on the action the agent intends to take. And something decides which action to take, using the prediction to choose. In the 2018 "World Models" paper that gave the term its modern meaning, David Ha and Jürgen Schmidhuber labelled these the vision, memory, and controller components, and trained the controller entirely inside the memory component's imagination — the agent learned to drive a race car by dreaming about driving, not by driving.

The reason this is worth the trouble is sample efficiency. Real robots are slow, expensive, and break. A simulation is cheap, fast, and can be run a million times. If the simulated world is faithful enough, everything you learn inside it transfers out.

The analogy: your kitchen in the dark

You have a world model of your own kitchen. Walk into it at night without the lights and you know roughly where the counter is, that the mug near the edge will tip if you knock it, and that the drawer sticks if you pull it fast. You know these things because you have pushed, spilled, and yanked thousands of times — not because you memorised a photograph of the room.

That's the distinction people miss. A photograph is a description of the world. A world model is a prediction machine: you supply an action, and it tells you what changes. A system that has only seen video of mugs being knocked over can produce a gorgeous clip of a mug falling and still have no idea that it could knock the mug over — or that catching it is an option.

What a world model is not

Not every video generator is one. OpenAI's February 2024 report on Sora framed video models as "world simulators," and that framing stuck — but it's contested. Yann LeCun has argued for years that predicting in abstract representation space is the tractable path and that generating every pixel is a detour, a position laid out in his 2022 paper A Path Towards Autonomous Machine Intelligence. Today "world model" covers video generators, compressed dynamics models, and representation-predicting architectures in the LeCun lineage. Two labs can use the phrase and describe incompatible objects.

It is not physics. A learned model approximates patterns in its training data. Where the data is thin, the predictions drift into physically impossible scenes, and a long rollout compounds the error.

It doesn't have to be perfect to be useful. DeepMind's DreamerV3 learned a world model from pixels and used it to become the first system to collect diamonds in Minecraft without human demonstrations. MuZero beat expert humans at Go, chess, and Atari without ever being told the rules of any of them — it learned the dynamics itself. Neither model was a flawless simulator; both were good enough to plan inside.

Why robots are the hard case

For a robot, "what happens next" splits into two questions: what will the world do, and what will the world do because of what I did. A conveyor belt moves on its own; a gripper changes things by grabbing. Mix them together and a model can predict the next frame accurately while having no idea which changes it caused — which means it can't answer the only question a controller actually asks: if I do something different, does it come out better?

That "action-conditioned" problem is the live frontier, and it's not solved. A August 2026 paper testing today's robotic world models found they follow expert actions reasonably well but fall apart on actions nobody demonstrated — they ignore the commanded action or generate nonsense. We went deeper on the companies racing to fix this in Deep Dive — Robots are learning what their actions do, not what to do, and on the commercial stakes in Unitree's founder puts a number on robotics: two '80%' thresholds.

Where to start

Ha and Schmidhuber's original paper is short and its interactive companion site is unusually accessible. LeCun's position paper is the best statement of the skeptical case. For where robotics has taken the idea since, a recent tutorial surveying the jump from world models to "world action models" is the cleanest map.

Related reading: What is an AI agent? on the systems that would use one · What is continual learning? on what it takes to keep updating it · What is a transformer? on the architecture most of them are built from.

If a world model can dream a good enough simulation, does a robot ever really need to fail in the real world to learn? Tell us in the comments.

Sources: Ha & Schmidhuber — World Models (arXiv) · Hafner et al. — Mastering Diverse Domains through World Models (arXiv) · LeCun — A Path Towards Autonomous Machine Intelligence (arXiv) · OpenAI — Video generation models as world simulators · From World Models to World Action Models: A Concise Tutorial for Robotics (arXiv) · Do Robotic World Models Really Follow Actions? (arXiv)