Mixture of experts
Mixture of experts (MoE) is a way of building an AI model so that only part of it runs for each piece of input. The model contains many smaller sub-networks, called experts, and a router picks a few of them to process each token. The rest sit idle for that token.
Why it matters
MoE lets a model hold a lot of knowledge while doing less work per token. A model can have a very large number of parameters in total but use only a small fraction of them for any one token. You get much of the quality of a very large model at closer to the speed and cost of a smaller one.
That’s why many recent models use it. Mixtral and the DeepSeek models are well-known open-weight examples.
Total vs. active parameters
MoE model specs usually give two numbers, a total parameter count and an active one. Total parameters is the size of the whole model, which sets how much memory you need to load it. Active parameters is how many are used for each token, which roughly sets the speed and compute cost.
The gap between the two catches people out. An MoE model with a large total still needs a lot of GPU memory to run, even if each token only touches a small part of it. So these models are cheap to serve at scale, but not always easy to run on your own hardware.
What “expert” means here
The name is a little misleading. The experts don’t line up with subjects like law or biology. During training the router learns which experts work well for which tokens, and the specializations that come out of that are often hard for people to interpret.
What it means for you
For most people using AI at work, MoE is an implementation detail. You’ll notice it indirectly, as lower prices and faster responses from capable models that would otherwise be expensive to serve.