Ask a large language model to explain photosynthesis to a ten-year-old, translate a legal clause into Portuguese, or find the bug in a snippet of Python, and it will usually do a creditable job of all three. It is tempting to imagine something elaborate inside: a database of facts, a grammar engine, a separate module for code. The reality is both simpler and stranger. At its core, the model was trained to do one thing — predict what comes next in a piece of text — and almost everything else emerged from getting very, very good at that game. Understanding how that works is the single best defence against the two common misreadings of these systems: that they are “just autocomplete”, and that they are something close to magic.
Text as tokens
A language model does not see letters or words the way we do. It sees tokens: chunks of text drawn from a fixed vocabulary, typically tens of thousands to a few hundred thousand entries long. Common words are often a single token; rarer words are split into pieces; spaces and punctuation get folded in along the way.
This detail explains some famous oddities. Models have historically stumbled on tasks like counting the letters in a word, because they never saw the letters individually — they saw a couple of opaque tokens. Arithmetic on long numbers can suffer similarly, depending on how digits are chunked. When a model seems oddly clumsy at something a child could do, tokenisation is often part of the story.
Attention: letting every word look at every other
Each token is turned into a long list of numbers, a vector, that represents it. The breakthrough architecture behind modern models, the transformer, introduced in the 2017 paper “Attention Is All You Need”, processes those vectors through many stacked layers. The crucial ingredient in each layer is attention.
Attention lets each position in the text gather information from every earlier position, weighting some far more heavily than others. In the sentence “The trophy did not fit in the suitcase because it was too big”, working out what “it” refers to requires connecting “it” to “trophy” and reasoning about size. Attention provides the machinery for such connections to be learned. Stack dozens of layers, each refining the representation, and the model can build up rich, context-sensitive pictures of what the text means and where it is going.
At the very end, the model outputs a probability for every token in its vocabulary: how likely each one is to come next.
Two phases of training
Pretraining: reading a library
In pretraining, the model is shown an enormous corpus of text — web pages, books, code, reference material — and repeatedly asked to predict the next token. When it is wrong, its billions of internal parameters are nudged slightly to make the right answer a little more likely. Repeat this across trillions of tokens.
Here is the key intuition. To predict the next word of a physics textbook well, it helps to have absorbed some physics. To predict the next line of a working program, it helps to model what the code does. To continue a dialogue, it helps to track who knows what. Nobody programs these abilities in. The prediction objective simply rewards whatever internal structure makes predictions better, and it turns out that a great deal of structure about the world is useful for predicting text about the world.
Post-training: shaping an assistant
A freshly pretrained model is a strange creature. Give it a question and it might continue with three more questions, as if it were completing a quiz. Post-training turns it into a helpful assistant. This typically involves fine-tuning on examples of good responses, and then techniques such as reinforcement learning from human feedback, where people (or models guided by written principles) compare outputs and the model is steered toward the preferred ones. More recent methods also reward models for reaching verifiably correct answers on problems like maths and code.
Post-training does not usually teach much new knowledge. It shapes behaviour: format, tone, willingness to decline harmful requests, the habit of reasoning step by step before answering.
Sampling: choosing the words
When generating, the model does not simply pick the single most likely token every time. It samples from the probability distribution, and a setting often called temperature controls how adventurous that sampling is. Low temperature makes output more predictable; higher temperature makes it more varied. Each chosen token is appended to the text, and the whole process repeats, one token at a time, until the response is done.
This is why asking the same question twice can give different answers, and why a model can talk itself into a corner: once a token is written, every later prediction is conditioned on it.
What the mechanism explains
Keeping the mechanism in mind makes much of a model’s behaviour less mysterious.
- Fluency. Models are superb at producing text that sounds right, because sounding right is precisely what the training objective measures most directly.
- Confident errors. The same fluency applies when the model lacks the relevant knowledge. A plausible-sounding citation or statistic is exactly the kind of text the model has learned to produce. Nothing in the basic mechanism guarantees that plausibility and truth coincide, which is why so-called hallucinations persist even as they become rarer.
- Sensitivity to phrasing. The model’s output is conditioned on every token of the input. Change the wording, the examples or the order of instructions and you change the context it is predicting from — sometimes dramatically.
- Knowledge cutoffs. A model knows what was in its training data and nothing after. Unless it is connected to search or other tools, it cannot know about events after its training ended, and it may not reliably know where that boundary lies.
What remains genuinely unclear
It would be satisfying to end the story there. But honesty requires admitting how much is still unsettled.
Why capabilities emerge as they do is only partly understood. Researchers have documented reliable relationships between scale and prediction quality, yet specific skills sometimes seem to appear rather abruptly, and there is ongoing debate about how much of that apparent suddenness reflects the models and how much reflects the way we measure them.
What is happening inside is an open research frontier. The field of interpretability has made real progress — identifying internal features that correspond to recognisable concepts, and tracing some of the circuits models use for particular tasks. But a full account of how a large model arrives at a given answer remains out of reach. We can build these systems far more precisely than we can explain them.
Whether models “understand” is partly an empirical question and partly a philosophical one. Models clearly build internal representations that go well beyond memorising surface patterns; they generalise to problems they have not seen. They also fail in ways that seem alien, confidently botching tasks slightly outside familiar territory. Thoughtful researchers disagree about how to describe this, and part of the disagreement is simply that “understanding” was never a precise word to begin with.
Beyond both caricatures
“Just autocomplete” is technically accurate about the training objective and badly misleading about the result, a bit like describing evolution as “just differential reproduction”. The simplicity of the rule tells you little about the complexity of what it can produce. But the opposite error is equally tempting: to treat fluent language as evidence of a mind like ours, with our kind of grounding in the world and our kind of reliability.
The most useful stance sits between them. A language model is a system that has compressed an astonishing amount of the structure in human text into a machine for continuing it. That machine can reason, explain and create in ways that are genuinely useful, and it can be wrong in ways that are genuinely hard to spot. Knowing that both things come from the same simple mechanism is where working with these systems wisely begins.