World models that ignore what people believe predict the wrong actions
A new "Mental World Modeling" framework adds beliefs, intentions, and social norms to world models — and even weak language models running it beat frontier giants answering directly.
Current world models simulate physics but skip the people in the scene — and a new paper argues that omission makes them predict human behavior badly. Systems like Sora, Genie 3, JEPA, and Marble track objects, motion, and occlusion, but nothing about what the humans involved believe, want, or consider socially acceptable. The researchers' example is disarmingly simple: move someone's cup into a cabinet while they're not looking. A purely physical model sees a perfectly correct scene — and predicts the wrong next action, because only a model tracking the person's belief about where that cup is can explain what they'll do next.
The paper proposes Mental World Modeling (MWM): extend the world state with mental variables — beliefs, attention, goals, intentions, emotions, social norms, relationships. Every action gets split into a physical carrier (speaking, pointing, grasping) and a mental payload (comforting, deceiving, rejecting). The same gesture of sliding a cup across a table can be an apology, a deception, or an act of care; without the mental variables, a model can't tell which.
The reference implementation, MENTIS, is a training-free pipeline with no fine-tuning anywhere. It parses the scene, renders the egocentric view the person actually sees, splits candidate actions into physical and mental components, simulates each outcome, then scores every branch on three axes: physical plausibility, mental consistency, and social appropriateness before picking one deterministically. Every stage writes machine-readable intermediates, so an error traces back to a specific step.
Evaluation used a new benchmark, Menti-Bench — 448 decision scenes spanning text, picture stories, and sound-video clips, each with six response options and human-written references documenting the underlying mental states. Across eight language models tested, accuracy climbed with every added layer: direct answers averaged 63.3 F1, six-way self-consistency reached 77.9, the full pipeline hit 87.9. Humans score 98.5 under the same protocol. The kicker: GPT-4.1 running the full MWM pipeline (84.9) beats GPT-5.6-Sol answering directly (83.6) — structure beats raw scale here. Removing the mental channel costs an average of 12.1 F1 points; removing the physical channel costs 16.5.
The ablations point to where this field's real bottleneck sits: about 80 percent of the remaining gap to humans comes from errors in predicting how coupled physical-mental states transition over time — describing the current state is nearly solved; simulating how it evolves is not. That lands in the middle of an unresolved debate about what a world model even is — Peking University-led researchers recently proposed excluding text-to-video generators entirely, and Yann LeCun has long called the generative approach a dead end. This paper's critique applies to all of them at once: they don't just disagree on architecture, they're missing a whole state dimension.
Our take: this reframes theory-of-mind work from a model capability into a systems problem. If a training-free wrapper lets a small model outpredict a frontier giant on human behavior, the value isn't in scale — it's in state design, and that's cheap to iterate on.
What to watch: whether the next wave of agent frameworks bolts mental variables onto their planner loops — and whether anyone couples MWM-style transitions into robot policies, where misreading a human's belief is a safety bug, not a benchmark loss.
If your agents had a live model of what you believe versus what's true, would you trust them more or less? Tell us in the comments.
Sources: The Decoder · Mental World Modeling — MENTIS (GitHub)
