For most of the modern LLM era, the recipe was stable: pre-train on a very large corpus, fine-tune on curated demonstrations, align with human preference feedback. Capability came from scale, and the model produced its answer in a single forward pass through the tokens. Reasoning models broke that pattern, and understanding why requires being precise about what the old recipe could and could not do.
The standard pipeline, briefly
Pre-training is next-token prediction over a very large corpus. It produces a model that has absorbed an enormous amount of the structure of language and, incidentally, of the world described by that language. It is not yet useful as an assistant.
Supervised fine-tuning narrows that raw model onto demonstrations of the behaviour you want — instructions followed, questions answered in a particular register. It is cheap relative to pre-training and it does most of the work of turning a text predictor into something you can talk to.
Alignment — RLHF, and later cheaper variants such as DPO — tunes the model against human preferences between candidate responses. It sands down the behaviours that are technically fluent but unhelpful, unsafe, or simply annoying.
The problem this recipe cannot solve
Consider a genuinely hard problem — a competition mathematics question, a multi-step logic puzzle, a subtle bug spread across several files. A model trained this way must commit to its first token immediately and produce the answer left to right, with no facility for backtracking. If the third step of a twelve-step argument goes wrong, everything downstream inherits the error and the model has no mechanism for noticing.
Chain-of-thought prompting was the first serious patch. Asking the model to work step by step genuinely helps, because it lets intermediate results occupy the context window where subsequent tokens can attend to them. The model is, in effect, using its own output as scratch memory. But the reasoning it produces is still a single unbacktracked pass. It is a better-organised guess.
Sampling techniques went further. Generate many chains, then take a majority vote or score them with a verifier. This works, and the fact that it works is the important clue: the model is frequently capable of the correct reasoning and merely fails to produce it reliably on the first attempt. The capability is latent in the weights; what is missing is a search procedure that finds it.
Why this is a training problem, not a prompting one
Every technique above is applied at inference time to a model that was never trained to reason. The model learned to imitate text that happened to contain reasoning; it did not learn to reason under an objective that rewards reaching correct conclusions.
That distinction is the whole story. Imitation learning is bounded by its demonstrations. A model trained to reproduce expert text can approach expert performance but has no route past it, because nothing in the objective rewards finding a better path than the one in the training data. Reinforcement learning has no such ceiling — it optimises against outcomes, and outcomes can be scored on problems where no human demonstration exists.
The obstacle was always the reward signal. RL needs to score intermediate states, and for open-ended text there is no principled way to say how good a half-finished argument is. Verifiable domains — mathematics, code that compiles and passes tests, formal proofs — are the exception, and that is precisely where reasoning models first showed dramatic gains. It is not a coincidence.
Part two takes up what happens when you actually apply that idea: what it borrows from game-playing systems, and what it costs.
