07 / experts
A committee where only two speak
The biggest models are not one giant brain. They are a panel of specialists, and for every single token a tiny router picks just two of them to do the work.
Eight desks, one dispatcher
A few chapters back you met the MLP, the big blender that reworks each token's meaning. In a frontier model that blender gets so large that running all of it for every token would be absurd. So it gets sliced into independent specialists called experts, with a dispatcher out front: the router. Each arriving token is scored against all eight experts, sent to the top two, and handed back as a weighted blend of their answers.
Try it. The panel below is a miniature: eight experts with hand-labeled tastes and a router that really does the scoring math.
Send bonjour, then 42. Same tray, very different routes. Hover or tab across the stations to read their specialties.
Nobody handed out these jobs
The router is not programmed, and neither are the specialties. The router is itself a small learned circuit, trained alongside everything else. Experts drift toward whatever division of labor makes prediction cheapest, and the split that emerges is rarely the tidy org chart a human would draw. Instead of a sports expert and a cooking expert, you tend to get experts for textures: one that perks up for punctuation, one for digits and units, one for the connective tissue between words.
Our eight nameplates are honest caricatures. Real experts have no labels at all, just habits you can only discover by watching which tokens the router keeps sending them. What is faithful here is the mechanism: real scores, a real softmax, the top two chosen, their answers blended in proportion. And every token takes its own route. In one sentence, Paris and the comma right after it may be handled by entirely different circuitry.
The bill
Why bother with all this dispatching? Because of what it does to the bill. A dense model runs its entire MLP for every token, no exceptions. A sparse mixture-of-experts model keeps the same shelf of knowledge, all eight experts' worth of parameters, but each token only pays for the two it visits. Watch the meters.
Run the stream, then switch to top-1 or top-4. The counter recomputes live. At top-8, sparse quietly becomes dense.
The number that matters
This is the trick that lets frontier models get enormous without getting slow. Total parameter count tells you how much the model knows: the whole shelf, every expert included. But speed and cost per token are set by the active parameters, the slice that actually fires. A model can advertise hundreds of billions of parameters while each of your tokens only ever touches a fraction of them.
So when two models boast the same headline size, ask the follow-up question: how much of that wakes up per token? A committee where only two speak can afford to be a very large committee.