How LLMs Work

TL;DR — A language model does exactly one thing: given the tokens so far, output a probability for every possible next token. Text appears because you take one of those tokens, append it, and ask again — that loop is what "generating" means, and it's called autoregressive. The model in §5 is forty lines of counting and it produces a fluent sentence it was never taught the grammar of, which is the point: fluency comes from statistics over sequences, not from understanding. That same mechanism explains the failures. A model always returns a distribution, so an unfamiliar context yields a confident guess rather than a blank — there is no "I don't know" state to fall into. And picking the most likely token every time hides the spread, which is why the same prompt can be certain and wrong. It stops being a useful model of the system once you need to explain why a real model is good at reasoning; counting explains the shape of the output, not the capability.

---

1. Simple explanation

Ask a language model what it does and the honest answer is disappointingly narrow: it reads a sequence of tokens and predicts what comes next. Not a sentence, not an answer — one token.

"Predicts" is loose. What it actually returns is a number for every token in its vocabulary: the probability that this one comes next. For a vocabulary of 100,000 that's 100,000 numbers, most of them nearly zero.

Then something outside the model picks one, sticks it on the end, and asks again. And again. That loop is where paragraphs come from. The model has no plan and no draft; each token is chosen with the previous ones fixed and nothing after them decided.

Analogy — the world's best autocomplete, with no idea where it's going. Your phone suggests the next word from what you've typed. A language model does the same over a much longer history with a much better sense of what follows what. Crucially, autocomplete doesn't know your sentence's destination — it just knows what usually comes next. The analogy carries the mechanism precisely, including the failure: autocomplete never says "I have no idea." It always has a most-likely next word, so it always offers one, whether or not the context makes sense.

---

2. Diagram

WHAT ONE CALL TO A MODEL DOES

  tokens so far:  ["the", "cat", "sat", "on", "the"]
        │
        ▼
  ┌────────────────────────┐
  │        the model       │
  └───────────┬────────────┘
              ▼
   a probability for EVERY token in the vocabulary
        0.67  'mat'
        0.33  'rug'
        0.00  'xylophone'
        ...   (99,997 more, all near zero)

  That is the whole output. No sentence. One distribution.


GENERATION = THAT, IN A LOOP  (autoregressive)

  step 0  ["<s>"]                        -> 'the'    ┐
  step 1  ["<s>","the"]                  -> 'cat'    │ each output is
  step 2  ["<s>","the","cat"]            -> 'sat'    │ appended and fed
  step 3  ["<s>","the","cat","sat"]      -> 'on'     │ straight back in
  step 4  ... "on","the"                 -> 'mat'    │
  step 5  ... "the","mat"                -> '</s>'   ┘ stop
                                            ▲
                                    'stop' is just another token


WHY IT NEVER SAYS "I DON'T KNOW"

  familiar context     ▁▁▁▁▁█▁▁▁   one token dominates -> confident, right
  unfamiliar context   ▁▁▂▂▂▂▂▁▁   spread out          -> confident, WRONG

  Both return a distribution. Both have a top token. The model
  cannot return nothing, so there is no failure state to detect.

---

3. How it works

3.1 The only operation

Everything a model does reduces to one function:

  tokens_so_far  →  probability for each possible next token

Chat, summarising, translation, code — all of it is that function called repeatedly. "Answer the question" is not a separate capability; it's what you get when the most likely continuation of a question happens to be an answer, because that is what the training text looked like.

Tokens, not words, are the unit — see Tokenization.

3.2 Autoregressive generation, and what it costs you

Take a token from the distribution, append it, call again. Two consequences follow directly from that loop.

Output is serial. Each token needs the previous one, so generation cannot be parallelised the way reading the input can. That is why output tokens dominate latency and why capping output length is the strongest latency lever you have.

Mistakes compound. Once a token is emitted it becomes part of the context for everything after it. There is no backtracking. A wrong turn at token 5 is a premise for tokens 6 onward, which is why a model that starts down a wrong path continues it fluently rather than correcting.

Stopping is also just prediction: a special end-of-sequence token becomes the most likely continuation, and the loop halts. The model doesn't decide it's finished; "finished" is a token like any other.

3.3 Where the probabilities come from

The toy in §5 uses counting: look at every place this context appeared in the corpus and tally what followed. That's an n-gram model, and it's genuinely how the earliest language models worked.

Real models don't store counts — they learn parameters that map a context to a distribution, and the mechanism is attention over the whole sequence rather than an exact-match lookup on the last two tokens. The differences that matter:

The first row is the important one, and §3.5 is about it. But the output type is identical — a probability per token — and so is the generation loop. That's why the toy is worth reading even though it isn't the real thing.

3.4 Training, briefly

Pretraining is the same next-token task run over enormous amounts of text: show the model a prefix, compare its prediction to the actual next token, nudge the parameters, repeat. No labels are needed because the text is its own answer key.

Then instruction tuning teaches it that a question should be followed by an answer rather than by more questions, and preference training nudges it toward responses people rate well. Neither adds a new mechanism — both reshape the same distribution.

The consequence worth remembering: the knowledge is frozen at the end of training, which is why anything recent or private has to be supplied in the prompt. That's the entire argument for retrieval — see What RAG Is and When to Use It.

3.5 The failure this explains: there is no "I don't know"

A trained model always produces a distribution over its whole vocabulary. Every context, however unfamiliar or nonsensical, yields a set of probabilities with a top entry.

So the model has no state corresponding to "nothing follows from this." It cannot return empty. When it doesn't know, the distribution is flatter and the top token is less dominant — but code that takes the top token gets a token either way, and the text reads identically. Confidence in the output is not confidence in the answer.

This is why hallucination isn't a bug to be patched: it's what a next-token predictor does when the context doesn't constrain the answer. The mitigations all come from outside the model — supply the facts (retrieval), check the answer against a source (grounding), or refuse when retrieval found nothing. Hallucination Detection & Grounding is the depth.

3.6 Where this model of the model stops helping

Counting explains the shape of the output — a distribution, a loop, no null state. It does not explain capability. Why a large model can follow a multi-step instruction, write working code, or hold a constraint across paragraphs is not answered by "it predicts the next token", any more than "neurons fire" explains a conversation. The mechanism is right and the explanation is incomplete, and anyone claiming next-token prediction proves models can't reason is overreading a true statement.

It also says nothing about attention, embeddings, or why scale helps — see Transformers Deep Dive — and nothing about multimodal models, where images and audio enter as tokens of a different kind.

---

4. The math

4.1 The objective

  the model estimates    P(next_token | all preceding tokens)

  a whole sequence is the product of those steps:
      P(t1..tn) = P(t1) * P(t2 | t1) * P(t3 | t1,t2) * ... * P(tn | t1..tn-1)

  and training minimises the negative log of that on real text
  (cross-entropy: see foundations/1-2/logarithms-and-exponents)

Working in logs matters practically: multiplying thousands of numbers below 1 underflows, so everything is done as sums of logarithms.

4.2 The counting estimator

  P(w | context) = count(context followed by w) / count(context)

  with the fatal gap:  count(context) == 0  →  undefined

That gap is the one row in §3.3's table that separates the toy from reality. A trained model never divides by zero because it never looks anything up.

4.3 Worked example

Count which token follows each two-token context in five short sentences about cats and dogs.

What the model outputs — not text, a distribution:

  after 'the cat'          -> 0.50 'sat'  0.25 'looked'  0.25 '</s>'
  after 'cat sat'          -> 1.00 'on'
  after 'sat on'           -> 1.00 'the'
  after 'on the'           -> 0.67 'mat'  0.33 'rug'

Read the first and last lines. After the cat, three continuations are genuinely plausible and the model says so. After on the, two are. The model is not confused — it is reporting a real spread.

Generation is that in a loop, taking the top token each time:

  step 0: 1.00 'the'      sentence so far: the
  step 1: 0.60 'cat'      sentence so far: the cat
  step 2: 0.50 'sat'      sentence so far: the cat sat
  step 3: 1.00 'on'       sentence so far: the cat sat on
  step 4: 1.00 'the'      sentence so far: the cat sat on the
  step 5: 0.67 'mat'      sentence so far: the cat sat on the mat
  step 6: 1.00 '</s>'  -> stop

  result: 'the cat sat on the mat'

A grammatical English sentence, produced by a program containing no grammar, no nouns, and no concept of a cat. It counted pairs.

Notice step 2: it committed to sat at probability 0.50 — a coin flip — and everything after that was conditioned on the choice. looked was equally available and would have produced a different, equally valid sentence. Taking the top token hides that the decision was nearly arbitrary.

4.4 And the failure, in the same model

  context never seen: no prediction possible

The toy honestly returns nothing for the elephant, because there are no counts. A real model cannot do this — it returns a distribution for every context, so the equivalent case produces a confident-looking guess. The toy's most obvious limitation is precisely what makes real models unreliable.

---

5. Real code

"""Next-token prediction, built small enough to read: counts, not magic."""
from collections import Counter, defaultdict

CORPUS = """
the cat sat on the mat
the cat sat on the rug
the dog sat on the mat
the dog barked at the cat
the cat looked at the dog
"""


def train(text: str, order: int = 2) -> dict[tuple[str, ...], Counter]:
    """Count which token follows each context of `order` tokens. That is the model."""
    model: dict[tuple[str, ...], Counter] = defaultdict(Counter)
    for line in text.strip().splitlines():
        toks = ["<s>"] * order + line.split() + ["</s>"]
        for i in range(order, len(toks)):
            model[tuple(toks[i - order:i])][toks[i]] += 1
    return model


def distribution(model, context: tuple[str, ...]) -> list[tuple[float, str]]:
    """Turn counts into probabilities -- the model's actual output."""
    counts = model.get(context)
    if not counts:
        return []
    total = sum(counts.values())
    return sorted(((n / total, tok) for tok, n in counts.items()), reverse=True)


model = train(CORPUS, order=2)

print("WHAT THE MODEL OUTPUTS: a probability for every possible next token")
for ctx in [("the", "cat"), ("cat", "sat"), ("sat", "on"), ("on", "the")]:
    dist = distribution(model, ctx)
    shown = "  ".join(f"{p:.2f} {t!r}" for p, t in dist)
    print(f"  after {' '.join(ctx)!r:<18} -> {shown}")

print("\nGENERATION IS THIS, IN A LOOP (always take the most likely token)")
ctx = ("<s>", "<s>")
out: list[str] = []
for step in range(9):
    dist = distribution(model, ctx)
    if not dist:
        break
    p, tok = dist[0]
    if tok == "</s>":
        print(f"  step {step}: {p:.2f} '</s>'  -> stop")
        break
    out.append(tok)
    print(f"  step {step}: {p:.2f} {tok!r:<10} sentence so far: {' '.join(out)}")
    ctx = (ctx[-1], tok)      # slide the window: THIS is 'autoregressive'
greedy = " ".join(out)

print(f"\n  result: {greedy!r}")
print("  Note the model was never taught grammar, cats, or sitting. It counted.")

print("\nTHE SAME MECHANISM EXPLAINS THE FAILURES")
unseen = distribution(model, ("the", "elephant"))
print(f"  context never seen: {unseen if unseen else 'no prediction possible'}")
print("    -> a real model never returns nothing; it always has SOME distribution,")
print("       so an unfamiliar context yields a confident-looking guess, not a blank.")

tie = distribution(model, ("the", "cat"))
print(f"\n  genuinely uncertain context 'the cat': {[(round(p,2), t) for p,t in tie]}")
print("    -> three continuations are plausible. Greedy picks one and hides the rest.")
print("       That hidden spread is what temperature and top-p expose.")

assert distribution(model, ("<s>", "<s>"))[0][1] == "the"
assert greedy.startswith("the cat sat on the")
# The model only ever produces a distribution over the NEXT token.
assert abs(sum(p for p, _ in distribution(model, ("the", "cat"))) - 1.0) < 1e-9
# An unseen context has no counts at all -- the honest version of "I don't know".
assert distribution(model, ("the", "elephant")) == []
print("\nall assertions passed")

# Output:
#   WHAT THE MODEL OUTPUTS: a probability for every possible next token
#     after 'the cat'          -> 0.50 'sat'  0.25 'looked'  0.25 '</s>'
#     after 'cat sat'          -> 1.00 'on'
#     after 'sat on'           -> 1.00 'the'
#     after 'on the'           -> 0.67 'mat'  0.33 'rug'
#
#   GENERATION IS THIS, IN A LOOP (always take the most likely token)
#     step 0: 1.00 'the'      sentence so far: the
#     step 1: 0.60 'cat'      sentence so far: the cat
#     step 2: 0.50 'sat'      sentence so far: the cat sat
#     step 3: 1.00 'on'       sentence so far: the cat sat on
#     step 4: 1.00 'the'      sentence so far: the cat sat on the
#     step 5: 0.67 'mat'      sentence so far: the cat sat on the mat
#     step 6: 1.00 '</s>'  -> stop
#
#     result: 'the cat sat on the mat'
#     Note the model was never taught grammar, cats, or sitting. It counted.
#
#   THE SAME MECHANISM EXPLAINS THE FAILURES
#     context never seen: no prediction possible
#       -> a real model never returns nothing; it always has SOME distribution,
#          so an unfamiliar context yields a confident-looking guess, not a blank.
#
#     genuinely uncertain context 'the cat': [(0.5, 'sat'), (0.25, 'looked'), (0.25, '</s>')]
#       -> three continuations are plausible. Greedy picks one and hides the rest.
#          That hidden spread is what temperature and top-p expose.
#
#   all assertions passed

Forty lines, five sentences of training data, and it produces grammatical English. Swap counting for learned parameters and the corpus for a large fraction of the internet, and the loop is unchanged — which is the useful thing to take away.

---

6. Real-world example

A team built an assistant to answer questions about internal systems. During testing someone asked about a service that had been decommissioned two years earlier and got a detailed, plausible answer: an owning team, a rough architecture, a config file path. All invented, all internally consistent, delivered in exactly the tone of the correct answers.

The reaction was to treat it as a defect and look for a setting to fix. There isn't one, and understanding why saved them weeks.

The model was asked to continue a context that looked like "here is a question about an internal service." Its training contains vast numbers of such passages, so the most likely continuation is a confident technical description. Nothing in the mechanism distinguishes "I have specific knowledge of this service" from "this is the shape of text that follows this kind of question." The output was correct next-token prediction and a wrong answer, simultaneously.

The two things that misled them were both about confidence. The answer's fluency was read as evidence of knowledge, when fluency only reflects that the token sequence was likely. And they'd assumed a model would produce something visibly degraded when it didn't know — but there is no such state, so an unfamiliar context yields the same polished prose as a familiar one.

What actually helped came from outside the model: retrieve the relevant document and instruct the model to answer only from it, and return "I don't know" when retrieval finds nothing. Notably, the same question on a bigger model produced a better-written fabrication. Scale improves fluency; it does not add a null state.

---

7. Interview questions companies actually ask

Q1. What does a language model actually output? A probability distribution over its entire vocabulary for the next token — tens of thousands of numbers, most near zero. Not a sentence, not an answer. Text appears because something outside the model picks a token, appends it, and calls the model again. That loop is what generation is.

Q2. What does "autoregressive" mean and why does it matter? Each generated token is fed back in as part of the context for the next one. Two consequences: output must be produced serially, so output length dominates latency in a way input length doesn't; and there's no backtracking, so an early mistake becomes a premise for everything after it, which is why models continue down wrong paths fluently instead of correcting.

Q3. Why do models hallucinate? Because a model always returns a distribution. There is no state meaning "nothing follows from this context", so an unfamiliar prompt produces a top token just like a familiar one, and the text reads identically. Hallucination isn't a defect on top of the mechanism — it's the mechanism running on a context that doesn't constrain the answer. The fixes are external: supply the facts, check the answer against a source, allow refusal.

Q4. If it's just predicting the next token, how can it reason? That's the honest limit of the explanation. Next-token prediction describes the output shape — a distribution, a loop, no null state — and does not explain capability. Predicting the next token well over a huge and varied corpus apparently requires internal structure that supports multi-step behaviour, and "it's just autocomplete" is as incomplete as "brains are just neurons firing."

Q5. How does a model know when to stop? It doesn't decide. A special end-of-sequence token becomes the most likely continuation and the loop halts. Stopping is prediction like everything else — which is also why an unusual context can fail to produce it, and why you set a maximum output length as a backstop.

Q6. What's the difference between pretraining and instruction tuning? Pretraining is next-token prediction over enormous general text — no labels needed, because the text is its own answer key. Instruction tuning then teaches the model that a question should be followed by an answer rather than more questions, and preference training nudges it toward responses people rate highly. Neither introduces a new mechanism; both reshape the same distribution.

Q7. Why is a model's knowledge out of date? Because the parameters are frozen when training ends, so anything more recent, private, or changeable simply isn't in there. And since the model can't tell the difference between knowing and guessing, it answers anyway. That's the whole case for retrieval: put the current fact in the context rather than hoping it was in the training data.

---

8. When to use / tradeoffs

This mental model is the right one when:

• Explaining why a model invented something
• Reasoning about latency — output tokens are serial, input is not
• Deciding between retrieval and fine-tuning
• Understanding why the same prompt gives different answers
• Debugging a model that won't stop, or stops early

Reach for a different explanation when:

• The question is why is it capable → transformers, attention, scaling
• The question is about images or audio → multimodal tokenization
• The question is about a specific model's quirk → that model's documentation

Honest limits. The model in §5 is an n-gram counter, and it differs from a real system in ways that matter beyond scale. It matches contexts exactly, so it generalises not at all; a trained model handles a context it has never seen because similar contexts produce similar internal representations. It sees two tokens where a real model weighs thousands. And its most instructive property — returning nothing for an unseen context — is the one thing real models cannot do, so the toy demonstrates the failure by lacking it rather than by exhibiting it. Treat §5 as an accurate account of the interface (distribution in, token out, loop) and a poor account of the engine. The compounding-error and no-null-state arguments are sound; anything about capability is out of scope here by design.

---

9. Summary + related articles

• A model does one thing: given the tokens so far, output a probability for every possible next token. Not a sentence — a distribution.
• Generation is a loop: pick a token, append it, ask again. That's "autoregressive".
• Two consequences: output is serial (so output length drives latency), and there's no backtracking (so early mistakes become premises).
• Stopping is just another token becoming most likely. The model doesn't decide it's finished.
• Forty lines of counting produced "the cat sat on the mat" — grammatical English from a program with no grammar. Fluency is statistics over sequences.
• At step 2 it committed to sat at 0.50 — a coin flip — and conditioned everything after on it. Greedy decoding hides that.
• There is no "I don't know" state. Every context yields a distribution with a top token, so unfamiliar input produces confident prose. That's why hallucination is the mechanism, not a bug.
• Knowledge is frozen at training. Anything recent or private must arrive in the context.
• This explains the output's shape, not the model's capability. "Just autocomplete" overreads a true statement.

Related:

• Tokenization — what the tokens are before any of this starts
• Sampling and Temperature — how the next token is actually chosen from the distribution
• Transformers Deep Dive — the architecture this article deliberately skips
• Context Fundamentals — how much history the model can condition on
• What RAG Is and When to Use It — supplying facts the parameters don't contain
• Hallucination Detection & Grounding — catching the failure §3.5 describes
• Reasoning Budgets: When Thinking Tokens Are Waste — what changes when the model generates hidden tokens first
• Logarithms, Exponents & the Log Scale — why the objective is written in logs
• Embeddings and Cosine Similarity — the other thing you can do with a learned representation of text

Resources

• Shannon, C. E. (1948) — A Mathematical Theory of Communication, Bell System Technical Journal — where predicting the next symbol as a model of language begins; the n-gram idea §5 implements originates here.
• Vaswani et al. (2017) — Attention Is All You Need, arXiv:1706.03762 — the architecture that replaced counting: https://arxiv.org/abs/1706.03762
• Radford et al. (2019) — Language Models are Unsupervised Multitask Learners (GPT-2) — the argument that next-token prediction alone yields general capability.
• Brown et al. (2020) — Language Models are Few-Shot Learners, arXiv:2005.14165 — what scaling the same objective produced: https://arxiv.org/abs/2005.14165
• Jurafsky & Martin — Speech and Language Processing (3rd ed. draft), the n-gram language models chapter — the counting model of §5, done properly, with smoothing: https://web.stanford.edu/~jurafsky/slp3/
• Karpathy, A. — Let's build GPT: from scratch, in code, spelled out — the bridge from §5's counting to a real transformer, in runnable steps: https://www.youtube.com/watch?v=kCc8FmEb1nY