AI 101 — What is a mixture of experts?

A mixture of experts is a way of building an AI model where, instead of one giant network doing all the work, you split it into many smaller "expert" sub-networks — and a tiny router decides, for each piece of input, which two or three of those experts actually run. Most of the model sits idle most of the time. That single trick is why the largest models shipping in 2026 can be hundreds of billions of parameters in size while still being cheap enough to serve at scale.

This is the architecture hiding behind names you keep seeing in the news. DeepSeek V3 and V4, Llama 4, Mistral's Mixtral, Qwen3-235B, Kimi K2, and (according to multiple credible reports) GPT-4 are all mixture-of-experts models. When a model card says "235 billion total parameters but only 22 billion active per token," that "active" figure is the mixture-of-experts line item. It is also why a model can fit a terabyte of weights on disk yet still respond at conversational speed.

The one-paragraph mental model

A normal large language model is dense — for every word it processes, every parameter participates. A mixture-of-experts model replaces some of its layers with a panel of smaller sub-networks (the experts), and adds a router in front of them. For each token, the router scores the experts and picks a small top-k — usually two. Only those two experts do any work for that token, and their outputs are combined and passed to the next layer. The total parameter count stays huge, the per-token cost stays small, and the model is still one neural network, trained end-to-end. The 1991 paper that introduced the idea called it "Adaptive Mixtures of Local Experts"; the modern form was crystallized in a 2017 paper by Shazeer and colleagues that introduced sparse gating — only firing a few experts per example rather than blending all of them.

A library, not a stadium

Picture a vast library with thousands of subject-matter experts sitting at their own desks — a tax specialist, a poet, a C++ reviewer, a recipe-writer. You walk in with a question. A librarian at the front desk glances at it, picks the two or three people best suited to help you, and sends you their way. The rest of the library keeps reading. That is a mixture of experts.

A dense model is the opposite. Imagine one omniscient librarian who personally answers every question in the building. That librarian is your friend, but they are expensive, slow when the building gets busy, and the only way to make them smarter is to literally stuff more knowledge into one brain.

The library version scales because adding more experts doesn't make any one question slower — the router can always pick from a wider menu, and the experts themselves can specialize more deeply. That is exactly what has happened in production: Mixtral 8x7B in 2023 ran eight experts per layer and picked two; modern models often run 64, 128, or 256 experts, sometimes with a few "shared" experts that always fire on top of the routed ones. The numbers in model cards ("8x7B", "235B-A22B") are the architecture speaking.

Why it matters right now

Three things have made mixture of experts the default at the frontier, not an academic curiosity.

It decouples capacity from cost. A 671-billion-parameter MoE can run at the compute cost of a roughly 37-billion-parameter dense model. That is how a single training run can produce a model with enough knowledge to feel expert at coding, math, and conversation without breaking the inference budget. When you see a model priced at a fraction of its size, this is almost always the reason.

It enables trillion-parameter systems. The 1.6-trillion-parameter Switch Transformer from Google (2021) was one of the first to make the case at scale. Today's open-weights releases take the same logic further: more experts, finer routing, and a few shared experts that always fire so the model still handles the basics. Routing has also become a research field of its own — recent work explores "Super Experts" (a small set of experts that carry most of the load) and routing strategies that learn to balance tokens across the panel.

It is now everywhere, including on your phone. Some of the smaller MoE models released this year are sized to run on a laptop or high-end phone. The architecture is also why a single frontier model can simultaneously serve a coding product, a chat product, and an API without three separate training runs.

Common misconceptions

"Mixture of experts is just an ensemble." It looks like one — many models, combined — but it isn't. A traditional ensemble runs every model on every input and averages the outputs. A MoE runs only the chosen experts per input, and the whole thing is trained as one model with a learned router. The router is part of the network, not a wrapper around it.

"Experts are specialists in the human sense." Tempting, but mostly wrong. An "expert" in a 235B-A22B model is a small feed-forward sub-network that has learned, through training, to be useful for certain patterns in the hidden representation of tokens. Some emergent specialization is real — certain experts do become better at code, certain others at non-English text — but the boundaries are statistical, not categorical, and they shift during training.

"Active parameters tell you how smart the model is." Active parameters tell you how much compute the model spends per token. Total parameters tell you how much capacity the model has to store knowledge. A 22-billion-active MoE is not "a 22-billion-parameter model with extra junk attached" — it is a different shape of system, and often a more capable one than a true 22-billion-parameter dense model on the same training budget.

"MoE is always cheaper." Inference is cheaper per token. But the entire model — every expert, routed or not — still has to fit in GPU memory. That is why MoE complicates deployment: serving infrastructure has to be smart about which experts to keep hot and how to route batches of tokens so popular experts don't become bottlenecks. The architecture trades compute for memory and engineering complexity.

How to go deeper

If you want to read the original ideas, the 1991 paper Adaptive Mixtures of Local Experts (Jacobs, Jordan, Nowlan, Hinton) is short and readable. The 2017 paper Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (Shazeer et al.) is the bridge to modern LLMs. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (Fedus, Zoph, Shazeer, 2021) is the cleanest explanation of how a single top-1 router changed the field. The Hugging Face blog Mixture of Experts Explained is a working engineer's walkthrough of the same material with code and diagrams.

To see the architecture in the wild, the model cards for Mixtral 8x7B and DeepSeek-V3 are the two clearest examples: they spell out the expert counts, routing choices, and active-parameter budgets in plain text.

Related reading: for the training-time companion to MoE, see our AI 101 — What is fine-tuning?; for what the routed token is actually operating inside, see AI 101 — What is a context window?; and for the run-it-yourself story that MoE makes newly possible on consumer hardware, see AI 101 — What is model quantization?.

Have you tried running a MoE model locally, or do you mostly meet them through APIs? Tell us in the comments.

Sources: Mixture of Experts Explained — Hugging Face · What is mixture of experts? — IBM · Mixtral of experts — Mistral AI · Outrageously Large Neural Networks (Shazeer et al., 2017) · Switch Transformers (Fedus et al., 2021)