What a large language model is

A large language model is a function that reads some text and predicts what comes next, one small piece at a time. That is the whole mechanism. It sounds far too simple to explain anything interesting, and the surprising fact of the last few years is that when you train this one trick on enough text, doing it well forces the model to learn a great deal about the world the text describes.

The whole job is guessing the next piece

Start with the plainest possible description and do not add anything to it yet. A large language model takes a stretch of text and produces a guess about what the next fragment of text should be. Give it the words “the capital of France is” and it will guess “Paris.” Give it “def add(a, b): return” and it will guess “a + b.” It appends its guess to the text, then runs again on the slightly longer text, guesses the next fragment, appends that, and repeats. A paragraph of output is not produced in one shot. It is produced one fragment at a time, each fragment chosen in light of everything written so far, including the model’s own earlier guesses.

The fragment it works in is called a token. A token is usually a bit smaller than a word: common words are a single token, and rarer words get split into a few. For now the only thing that matters is that the model reads and writes in these tokens rather than in whole words or single letters. The choice of unit has real consequences for cost and for how much text fits at once, but those consequences are the subject of a separate idea. Here, just hold the shape of the loop: read the tokens so far, predict the next one, add it, repeat.

That is genuinely all a large language model does at the mechanical level. There is no separate module that stores facts, no lookup table of answers, no reasoning engine bolted on the side. Everything the model appears to do, answering a question, writing a function, translating a sentence, is the same operation running over and over: what token comes next.

Why “just predicting the next word” is not a small thing

The instinct on first hearing this is to be unimpressed. Predicting the next word sounds like what your phone keyboard does when it suggests “morning” after “good.” That is the right starting point, and the difference is entirely one of degree, which turns into a difference of kind. Your phone’s suggestion is built from simple counts of which word tends to follow which. A large language model is asked to predict the next token well across essentially all the text a company can gather, and to do that well you have to get good at some very hard cases.

Consider what it takes to reliably predict the last token of these lines. “The murderer, it turned out, was the” and then a name that only makes sense if you followed the whole story. “Water boils at one hundred degrees” and then “Celsius,” which requires knowing a physical fact. “She translated the French word chien into” and then “dog,” which requires knowing two languages. “The function returns the sum, so calling it with two and three gives” and then “five,” which requires doing arithmetic. To predict the next token as well as possible across billions of examples like these, a model cannot just memorize which word follows which. There are too many combinations, most of which it has never seen in that exact form. It has to build internal machinery that captures the patterns underneath: grammar, facts, cause and effect, a rough model of how people write and think.

This is the key turn, so it is worth stating slowly. Understanding was never the goal that was programmed in. Nobody wrote rules for grammar or a database of facts. The goal was only ever to reduce prediction error. But the cheapest way for the training process to reduce prediction error, once the easy patterns are used up, is to develop something that behaves like understanding. Apparent comprehension is a side effect of being pushed very hard on a simple objective. That is why the plain description and the impressive behavior are both true at once.

The model was never taught to understand. It was taught to predict, and understanding turned out to be the most efficient way to predict well.

The guess is a distribution, not a single answer

There is one detail in the mechanism that explains a lot of the model’s behavior, so it is worth making precise. The model does not output a single next token. It outputs a probability for every possible token at once. Given “the capital of France is,” it does not simply say “Paris.” It produces something closer to a ranked list: Paris is very likely, a comma or a period is somewhat likely because the sentence could be phrased differently, and thousands of other tokens sit at tiny probabilities. The full spread of those numbers is called a probability distribution over tokens.

A separate step then picks an actual token from that distribution. It can always take the most likely one, which makes the model steady and a little repetitive, or it can sample, sometimes choosing a slightly less likely token, which makes the output more varied and more human. This is why the same prompt can give you different answers on different runs. The underlying prediction is the same distribution each time. The choice made from it can differ.

Thinking in distributions also explains the model’s confidence problems. When the distribution has one clear winner, the model is, in effect, sure. When the top several tokens are all roughly as likely as each other, the model is unsure, but nothing forces it to tell you that. It still picks a token and writes it in the same fluent voice as everything else. A confident sentence and a wild guess look identical on the page. The uncertainty was real inside the distribution and it was flattened away the moment a single token was chosen.

Where the knowledge actually lives

If the model has learned facts and patterns, a fair question is where they are kept. The answer is a set of numbers called parameters, also called weights. A parameter is a single number that controls how strongly one internal signal influences another. A modern large language model has billions of them, sometimes hundreds of billions. When people say a model has seventy billion parameters, they mean it is a formula with seventy billion adjustable numbers inside it.

Training is the process of setting those numbers. It works like this: show the model a piece of real text with the next token hidden, let it predict, compare the prediction to the token that actually came next, and nudge every parameter a tiny amount in the direction that would have made the right token more likely. Do that across an enormous amount of text, an amount that would take a person many lifetimes to read, and the parameters slowly settle into values that predict well. All of the model’s apparent knowledge is encoded in the final values of those numbers. There is no sentence stored anywhere saying “Paris is the capital of France.” There is only a pattern of weights that, when the right tokens go in, makes “Paris” the likely next token.

This is why you cannot open a model and read its facts, and why you cannot simply edit one fact without risk. The knowledge is not filed in labeled drawers. It is distributed across the whole network, smeared over billions of numbers that each take part in countless different predictions. It is closer to how a skill lives in a trained muscle than to how a fact lives in a book.

Why scale changed what these models can do

Two quantities grew over the last several years, and their growth is most of the story of why these models became useful. The first is the amount of training text. The second is the number of parameters. Both went up by large factors, and the striking finding was that quality kept improving as they did, in a fairly smooth way. A bigger model trained on more text predicts better, and past certain sizes it starts to handle tasks that smaller models simply could not, such as following multi step instructions or writing working code.

It helps to see why more of both should matter. More text means the model meets more of the rare patterns, the odd phrasings, the specialized vocabulary, the corners of knowledge that a smaller sample would miss entirely. More parameters means the model has the capacity to store and combine those patterns rather than being forced to blur them together. A small model on a huge amount of text runs out of room to hold what it sees. A large model on a small amount of text has room but nothing to fill it with. The gains come from pushing both up together.

This should not be read as magic or as a promise that growth continues without limit. It is an empirical pattern, observed in practice, with real costs attached: the computation required is immense, and there are open questions about how far the pattern holds. The honest summary is narrow and useful. Within the range that has been built so far, scaling the data and the parameters together has reliably produced models that predict better and, as a consequence, do more.

What it knows, and what it only seems to know

Because the model learned from human text, it carries an enormous amount of what is in that text: common knowledge, the shape of many programming languages, the conventions of dozens of human languages, the usual way arguments and stories and functions are structured. On anything well represented in its training, it is often right and often useful. That is real, and it is the reason these models are worth using.

But it is important to be precise about the nature of that knowledge, because the failures follow directly from it. The model learned what text tends to look like. It did not learn which text is true. Truth and plausibility are correlated in human writing, since people mostly write true things, so predicting plausible text gets you true text much of the time. They are not the same, though, and where they diverge the model follows plausibility. Asked about a fact it never clearly saw, it does not go quiet. It generates the most likely sounding continuation, which can be a confident, well formed, completely invented answer. This behavior has a name and a set of remedies that belong to a later idea; the point here is only that it is not a bug bolted on. It is the direct shadow of how the thing works.

A second limit follows from the same source. The model has no senses and no memory of a life. It never saw the objects the words refer to, never ran the code it writes, never checked a claim against the world. Everything it appears to know about reality, it knows through the way people have written about reality. That is a rich source and a real one, but it is secondhand, and it has the gaps and biases of the text it came from.

The knowledge is frozen at training time

Here is the single fact that matters most for using these models well, and the one that is easiest to forget because the model talks so fluently in the present. When training ends, the parameters stop changing. From that moment the model is fixed. It knows the world as the training text described it up to the day the data was collected, and not one day further. It does not read the news. It does not watch its own outputs and learn from them. It does not quietly absorb your last conversation into its weights. Ask it about an event after its cutoff and it will either say it does not know or, worse, invent something plausible.

This is why a chat can feel like it remembers you within a single session yet start blank the next time. Anything it seems to remember inside a conversation is just text that has been kept in front of it and fed back in as part of the input. Nothing about that conversation has entered the frozen weights. When the conversation ends, that text is gone unless something outside the model stored it.

The frozen quality is exactly why models on their own are a poor fit for a living, changing codebase. A repository changes every day. The model’s knowledge does not, and it never saw your private code at all. The way to give a fixed model current, specific facts is not to retrain it for each question but to place the relevant facts into its input at the moment you ask, so its next token prediction runs over your reality rather than only the general one it was trained on. Doing that reliably is the reason the rest of this hub exists, and it begins with the units the model actually reads, which is the subject of tokens and tokenization.