Where the knowing comes from

Reading as a workout

Pretraining is one game played at absurd scale: cover the next word, guess it, peek, adjust. No rules are written down. The model earns everything it knows by being wrong slightly less often, word after word, across trillions of words of text.

The model below is real and it starts out knowing nothing. Press read and it will read a tiny story, keeping real counts of which word follows which. Every bar you see is a real tally, built in your browser, right now.

Scale this up and something odd happens. To keep winning the guessing game on real text, mere word pairs stop being enough. Predicting the next word of "the boiling point of water is" requires knowing chemistry. Predicting dialogue requires a feel for people. The knowledge is never the goal; it is the residue of getting better at guessing.

The taste test

After all that reading, the model can continue any text, which is not the same as being helpful. It might answer your question, or write a plausible next paragraph of a forum argument. So there is a second phase: humans read pairs of answers and pick the better one, thousands of times. The model learns to lean toward whatever the raters kept picking.

For the next minute, you are the rater. Same question, two answers, choose the one you would rather receive. Do it three times and watch the taste meters move.

Notice what this phase does not do: it adds no facts. The model knew about clouds before you clicked anything. Your choices tuned its manner: warmer or stiffer, longer-winded or brisker. This is why people say fine-tuning teaches taste, not knowledge. The librarian read the library long ago; the taste tests teach it how to talk to you at the front desk.

The machine that must answer

Here is the uncomfortable part. The model's whole reflex is "produce the most likely next word". It cannot leave a blank. Ask it something with no true answer and the reflex fires anyway, fluently. A hallucination is not a glitch; it is a very good guesser doing its job with nothing true to guess.

"I don't know" is not a special escape hatch either. It is just another phrase, competing for probability like every other phrase. It wins only if training made it likely to win.

In the untrained setting, "New" wins because the internet is full of sentences like "the capital of New Zealand", and capitals named New Something. The shape of the guess is perfect; only the world is missing. That is what a confident hallucination looks like from the inside: probability with no referent.

The whole machine, in one breath

This was the last chapter, so step back and look at what you have built up. Text becomes tokens. Tokens become points on a map of meaning. Attention lets every word study the words before it. Layers of small judgments stack into deep ones. The stack ends in a guess about one next word. Experts split the work, clever caches and shortcuts make it fast, and squeezing the numbers makes it fit. Then training, the chapter you just played through, is where all of it gets its content: read everything, guess constantly, and let human taste do the final polish.

Nine ideas, snapped together like bricks. No ghost in the machine, which is somehow the more remarkable outcome: a fluent, occasionally over-confident guesser built out of counting and correction. Now when someone asks you how these things work, you can do better than a shrug. You can hand them this and say: go play.