The fundamental difference: tokens vs. states

The core distinction between LLMs and world models is what they predict:

LeCun's critique of LLMs: Yann LeCun has argued publicly for years that autoregressive LLMs are a fundamentally limited path to general intelligence: they are trained to predict the next word based on statistical patterns in text, not to predict the next state of the physical world based on causal understanding. When an LLM generates a paragraph about a ball rolling down a hill, it is not simulating physics — it is predicting which words typically follow other words in similar passages, with no internal model of gravity, friction, or momentum. World models are explicitly designed to close that gap, and LeCun leaving Meta to build one at AMI Labs is the clearest institutional bet yet that this critique is right.

The 2025–2026 world models race

In the span of about a year, world models went from a niche research topic to one of the industry's most-funded frontiers:

JEPA — LeCun's architecture: AMI Labs is built on Joint Embedding Predictive Architecture (JEPA), developed at Meta. Unlike LLMs that process tokens, JEPA-based models operate in abstract latent spaces and predict how the state of the world changes in response to actions. The key insight: predict in representation space, not pixel space — this avoids the exponential complexity of modeling every visual detail, focusing instead on the semantically meaningful changes.

Why world models matter — real applications

For students: where to start: World models are a fast-moving research area — most of the best material is in papers and open repos, not polished tutorials. Start with: (1) DreamerV3 (Hafner et al., arXiv Jan 2023; republished in Nature in 2025 as "Mastering Diverse Control Tasks through World Models") — still the most complete open-source world model for RL tasks; (2) Nvidia Cosmos on GitHub (NVIDIA/Cosmos) — open weights you can actually run; (3) DeepMind's Genie 3 technical blog post; (4) LeCun's 2022 position paper "A Path Towards Autonomous Machine Intelligence" — the theoretical blueprint behind AMI Labs and JEPA.

Practice questions

1. What is the difference between a model-free and model-based reinforcement learning agent? (Answer: Model-free: learns a policy (what to do) or value function (how good is each state) directly from experience, without modeling the environment dynamics. Simple but sample-inefficient — needs many environment interactions. Model-based: explicitly learns a transition model P(s' | s, a) (what happens when action a is taken in state s). Can plan by simulating future trajectories without real environment interaction. Sample-efficient but requires accurate world model. World models aim to give RL agents model-based efficiency.)
2. What is DreamerV3 and how does it use a world model? (Answer: DreamerV3 (Hafner et al., arXiv 2023, Nature 2025): learns a compact world model in latent space — a Recurrent State Space Model (RSSM) that predicts latent states from the current latent state and action. The agent is trained largely within imagined rollouts from this world model rather than by directly interacting with the real environment during policy training; environment interaction mainly updates the world model itself. This lets DreamerV3 master diverse tasks (Minecraft, robot locomotion, classic games) with far fewer real environment steps than model-free RL.)
3. Why are world models important for safety in autonomous systems? (Answer: An autonomous car without a world model must learn purely from real experience — including crashes. A car with a world model can: simulate thousands of dangerous scenarios internally without real risk. Test 'what if I miss the red light' in simulation before ever encountering it. Plan by rolling out multiple potential future trajectories and choosing the safest. Predict other agents' behaviors. Real-world failures are catastrophic; a world model allows safety-critical scenarios to be explored in imagination.)
4. How does the concept of a 'mental model' in cognitive science relate to AI world models? (Answer: Cognitive science: humans maintain mental models of physics, social relationships, causality, and others' mental states. We plan actions by mentally simulating their consequences. Johnson-Laird (1983): mental models are the basis of reasoning and language understanding. AI world models operationalise this: a neural network that represents environment dynamics enables planning by simulation. The connection is deep — both biological and artificial agents that model their environment before acting are more adaptive and efficient than reactive systems.)
5. What is a 'latent space world model' and why is it more efficient than pixel-space models? (Answer: Pixel-space world model: learns to predict future video frames at full pixel resolution — computationally expensive (high-dimensional output, each step generates thousands of pixels). Latent space world model: compress observation to compact latent representation via VAE/encoder, model dynamics in latent space (small vectors), decode only for visualization. DreamerV3's RSSM models a compact latent state (tens of dimensions). Planning and policy learning happen in this compact space — orders of magnitude fewer computations than pixel-space modeling.)