Text in, text out — but what happens between?
How LLMs actually work, from raw web text to a working assistant
A large language model does exactly one thing: given a sequence of tokens, it produces a probability distribution over what token comes next. Everything else — the fluency, the apparent knowledge, the helpfulness, and the failures — emerges from repeating that single operation, billions of times during training and once per word during a conversation.
This page takes that claim apart. You'll tokenize your own text, watch attention weights get computed from actual Query and Key vectors, drag a temperature slider and see the distribution collapse or spread, and follow the whole pipeline from a web crawl to a deployed chatbot. Nothing here is a black box — every visualization runs the real arithmetic in your browser.
Model builders publish very little, and the figures that do circulate come from different years, different models, and different measurement conventions. Numbers here are cross-checked across the sources listed at the bottom, and are chosen to be representative of the current generation rather than exact for any one system. Where sources genuinely disagree, the page says so.
Pre-Training
Pre-training is the expensive part: months of compute spent compressing a large fraction of the public internet into a fixed set of numbers. The output is not an assistant. It is a document simulator — a system that, shown any prefix of text, can produce a statistically plausible continuation of it.
Four things have to happen before that works. You need text, and a lot of it. You need a way to turn text into numbers. You need an architecture that can look at a sequence and decide which parts of it matter. And you need a training loop that nudges billions of parameters toward better predictions, one gradient step at a time.
Data collection1a
Practitioners broadly agree that data quality and diversity affect the final model more than almost any other single choice — more than clever architecture tweaks, and often more than raw parameter count. So the pipeline that produces the training corpus is not a preliminary step. It is a large part of the engineering.
A modern open corpus starts from Common Crawl, a nonprofit archive that has been crawling the web since 2007 and publishes a snapshot roughly monthly, each containing on the order of 2.7 billion pages. Raw crawl output is mostly unusable: navigation chrome, cookie banners, spam farms, boilerplate, and near-duplicate copies of the same article on forty domains. Getting from that to trainable text takes a sequence of aggressive filters, each of which throws away far more than it keeps.
Fetch raw HTTP responses from the open web. Everything downstream is subtraction.
15 trillion tokens is roughly 11 trillion words. A fast human reader manages perhaps 300 words a minute. Reading the corpus once, without sleeping, would take on the order of 70,000 years. The model does it in weeks, in parallel, across tens of thousands of accelerators.
Tokenization1b
Neural networks consume numbers, not characters. Tokenization is the bridge, and the design space has two obvious bad ends. Give every word its own ID and the vocabulary is unbounded — every typo, name, and compound is an unknown symbol. Give every character its own ID and the vocabulary is tiny but sequences become enormous, and attention cost grows with the square of sequence length.
Byte Pair Encoding splits the difference. Originally a compression algorithm, BPE starts with individual bytes and repeatedly finds the most frequent adjacent pair in the corpus, merging it into a new single token. Run that merge step tens of thousands of times and you get a vocabulary that spends short tokens on common words and falls back to fragments — ultimately to raw bytes — for anything rare. Nothing is ever out-of-vocabulary.
The learned result is linguistically sensible without anyone designing it that way. Related
words end up sharing a stem: run, run|ning, run|ner.
The model gets to reuse what it knows about the root across all of its inflections.
the and the
are different tokens.
| Tokenizer | Used by | Vocabulary | Note |
|---|---|---|---|
| r50k / GPT‑2 BPE | GPT‑2 | 50,257 | 50,000 merges + 256 byte tokens + 1 end-of-text token |
| cl100k_base | GPT‑3.5, GPT‑4 | 100,277 | Much better at code, whitespace, and non-English text |
| o200k_base | GPT‑4o and later | ~200,000 | Roughly doubles again; fewer tokens per unit of text |
The model never sees letters. It sees token IDs. Ask it to count the r's in "strawberry" and
you are asking a system that perceives roughly three opaque chunks to report on characters it
has no direct access to. Arithmetic suffers similarly — whether 127 arrives as
one token or as 12+7 depends on quirks of the merge table, not on
anything mathematical.
The Transformer architecture1c · core
This is the engine. Introduced in 2017 in Attention Is All You Need, the Transformer replaced the recurrent networks that came before it with a design that processes every position in a sequence simultaneously. That change is what made training at internet scale economically possible — recurrent models had to walk a sentence left to right, one step at a time, and could not exploit a GPU's parallelism.
The original paper described an encoder–decoder pair, built for translation: an encoder to understand the source sentence, a decoder to emit the target. Three families descend from it. Encoder-only models like BERT read whole sequences bidirectionally and are used for understanding tasks. Encoder–decoder models like T5 keep both halves. And the models this page is about — GPT, Claude, Llama — are decoder-only: they read only leftward, and generate one token at a time. Writing with blinders on, deliberately.
Input embeddings and positional encoding
Each token ID indexes into an embedding table, retrieving a vector of learned numbers — 512 dimensions in the original paper, 12,288 in GPT‑3. These are not assigned by hand. They fall out of training, and they arrange themselves so that tokens used in similar contexts end up near each other in the space. That is the whole of what "meaning" is, inside the model: a position in a high-dimensional space, defined entirely by distributional company.
But processing every position at once destroys word order, and the dog bit the man must not equal the man bit the dog. So a positional encoding is added to each embedding — in the original paper, a fixed pattern of sines and cosines at geometrically spaced frequencies, chosen so that each position gets a unique signature and so that relative offsets are recoverable through simple linear operations. Modern models often use learned or rotary variants, but the job is identical: stamp order into the vector.
PE(pos,2i)=sin(pos/10000^(2i/d)),
PE(pos,2i+1)=cos(pos/10000^(2i/d)) — computed live, not illustrated; that is why the
diagonal wave bands below are real, and why they run at lower frequency toward the right. What is
not real: the token embeddings. A trained model's embedding table has 12,288+ dimensions
per token, arranged by gradient descent over billions of examples. These 24-dimensional vectors
are deterministic pseudo-random numbers seeded from each word's spelling — present only to show
that some vector is there for the position signal to be added to, not what a real one
contains.
Self-attention: Query, Key, Value
Consider "The cat sat on the mat because it was warm." Resolving it requires reaching back across the sentence. Self-attention is the mechanism that makes that reach possible, and it works by having every token ask a question and every token advertise an answer.
Each token's embedding is multiplied by three learned matrices, producing three vectors:
- Query (Q) — what this token is looking for. "I'm a pronoun; I need an animate noun behind me."
- Key (K) — what this token offers as a match. "I'm a noun, and I'm animate."
- Value (V) — the content actually passed along when a match happens.
Compatibility between a query and a key is their dot product. Every query is scored against
every key, the scores are divided by √d_k to keep them in a range where softmax
has usable gradients, and softmax turns each row into a set of weights summing to one. The
output for a position is the weighted sum of all Value vectors — mostly the content of
whichever tokens it scored highest.
Decoder-only models add one more thing: causal masking. Before softmax, every score for a future position is set to −∞, which softmax maps to exactly zero. A token can attend to itself and everything before it, never ahead. This is what makes training efficient — every position in a sequence can be trained to predict its successor in a single pass, without any position cheating by looking at the answer.
softmax(mask(QKᵀ/√d_k))V — nothing is
a hand-written attention value. Two honest simplifications: the weight matrices were
designed rather than learned by gradient descent, and the 16 embedding dimensions were
given interpretable meanings so you can read them. A real model discovers equivalent matrices
from data across billions of parameters, and its dimensions have no labels anyone wrote. The
arithmetic is the same operation, at the same place in the network.
Multi-head attention
One attention pattern per layer would be a severe bottleneck. A pronoun-resolution pattern and a subject–verb pattern are both useful, and they want to attend to different places. Multi-head attention solves this by running several attention operations in parallel, each with its own independent Q, K and V matrices.
The original Transformer used 8 heads over a 512-dimensional model, giving
each head 64 dimensions to work with — the model dimension divided by the head count, so total
compute stays roughly constant. Each head produces its own output; the eight outputs are
concatenated back to 512 dimensions and passed through a final projection matrix
W_O that lets the layer mix what the heads found.
When researchers inspect trained heads, they find genuine specialization: heads that track the previous token, heads that link pronouns to antecedents, heads that connect verbs to their subjects, heads that follow syntactic structure. Nobody assigned those roles. They are what gradient descent found useful.
Feed-forward, residuals, and layer norm
Attention moves information between positions. The feed-forward network processes each position independently — the same small MLP applied at every position, expanding to a wider hidden layer (2,048 dimensions against a 512-dimensional model in the original paper, a 4× ratio that most models still use), applying a nonlinearity, and projecting back down. It holds a large share of the model's parameters, and there is growing evidence it is where a lot of factual knowledge lives.
Two pieces of plumbing make deep stacks trainable. Residual connections add each sub-layer's input to its output, giving gradients an uninterrupted path back through the network and letting each block learn a refinement rather than a replacement. Layer normalization re-centres and re-scales activations to keep them in a stable numeric range. The original paper placed normalization after the residual addition; nearly every modern model moved it before the sub-layer instead, which trains more stably at depth.
From logits to a probability distribution
After the last block, the vector at the final position is projected by one more matrix into a score for every token in the vocabulary — roughly 100,000 raw numbers called logits. Softmax exponentiates them and normalizes, turning arbitrary reals into a genuine probability distribution.
That distribution is the model's complete output. Everything a language model produces is a sample from it, one token at a time, with each sampled token appended to the input for the next round.
| Transformer (2017, base) | GPT‑3 (2020) | Frontier (~2024) | |
|---|---|---|---|
| Layers | 6 | 96 | ~100–130 |
| Model dimension | 512 | 12,288 | ~16,000+ |
| Attention heads | 8 | 96 | ~100+ |
| Head dimension | 64 | 128 | ~128 |
| FFN hidden | 2,048 | 49,152 | ~4× d_model |
| Parameters | 65M | 175B | up to ~405B dense |
| Context window | ~512 | 2,048 | 128K – 1M+ |
Frontier-model specifications are not published. The right-hand column is inferred from open models of comparable capability and from what vendors have confirmed in passing. Treat it as an order-of-magnitude guide. One number that circulates widely and is wrong: several popular explainers cite "176 billion parameters" for ChatGPT — that appears to conflate GPT‑3's 175B with BLOOM's 176B, and in any case no current production model's count is public.
The training loop1d
Every parameter starts as a small random number, so the model's first prediction is uniform noise. Training is then a loop that is conceptually simple and computationally brutal:
- Sample a window of tokens from the corpus.
- Run it forward; get a predicted distribution at every position.
- Compare each prediction against the token that actually came next. The penalty is cross-entropy loss — the negative log probability the model assigned to the correct answer.
- Backpropagate to get a gradient for every parameter, and take a small step downhill.
- Repeat, for trillions of tokens.
The loss number has a concrete meaning. A model guessing uniformly across a 100,277-token
vocabulary scores ln(100277) ≈ 11.5. A well-trained model reaches roughly 2.4.
Since the loss is a negative log probability, that is the difference between assigning the
correct token a one-in-a-hundred-thousand chance and assigning it about one in eleven —
out of a hundred thousand candidates, on arbitrary text.
What that number buys is visible in the samples. Early on, output is character soup. Then spacing and word shapes appear, then grammatical fragments, then coherent sentences, then paragraphs that hold a topic. No one programs those stages; they emerge in order as the loss falls.
Loss falls predictably as a power law in three quantities: parameters, training tokens, and compute. The Chinchilla result added an important correction — for a fixed compute budget, earlier models were badly oversized relative to their data, and roughly 20 training tokens per parameter is closer to optimal. The striking part is that no clear plateau has appeared yet. That single empirical observation is most of the reason the field has spent the last several years building larger and larger training runs.
Inference and sampling1e
Generation is autoregressive: predict a distribution, pick a token, append it to the context, predict again. The model has no plan for the sentence it is writing and no ability to revise a token once emitted. Each step sees only what has already been produced.
Picking is where temperature enters. Before softmax, every logit is divided by a temperature T. Below 1, differences between logits are magnified, the distribution sharpens, and the model becomes conservative and repetitive — at the limit, always taking the single most likely token. Above 1, differences compress, the distribution flattens, unlikely tokens get real probability mass, and output becomes surprising and then incoherent. Most deployed chat systems sit somewhere around 0.7 to 1.0.
This is also why the same prompt gives different answers each time, and why "regenerate" works at all. The randomness is not a bug being tolerated; greedy decoding produces flat, loop-prone text, so sampling is doing real work.
purple, banana —
start winning draws.
The Base Model
Pre-training finishes and you have a base model. It is not a chatbot. Ask it a question and it may well answer with more questions — because in its training data, a question mark is frequently followed by another question. It has no notion of being helpful, no concept of a conversation, and no idea it is supposed to stop talking.
What it is, is an internet document simulator. Give it any prefix and it will produce a continuation drawn from the distribution of documents it absorbed. That turns out to be a remarkably powerful thing to be.
Few-shot learning
Show it three English→French pairs and it will translate the fourth — with no weight update and no training for translation. It is not learning; it has recognized the shape of the document as "a list of translation pairs" and is continuing that document faithfully.
Why it matters: prompting is a real interface into the model, not a workaround.
Memorization
Feed it the opening of a well-known Wikipedia article and it can often continue it close to verbatim. Nothing was looked up. The text is reconstructed from the weights alone — an artifact of that passage appearing often enough, in enough forms, to be stored rather than generalized.
Why it matters: knowledge lives in parameters, which is why the aggressive dedup step in 1a exists.
Confabulation
Ask about something after its cutoff, or something that never existed, and it produces a fluent, well-structured, entirely invented answer. The mechanism is unchanged — it is still sampling a plausible continuation. Plausibility is all it ever optimized for.
Why it matters: hallucination is not a malfunction. It is the normal operation of the machine on out-of-distribution input.
The "two files" mental model
Strip away the infrastructure and a deployed language model is, in principle, two files on disk. One holds the parameters: a large array of numbers. Llama 2 70B stored at 16 bits per parameter is about 140 GB. The other is the code that runs the forward pass — the arithmetic described in 1c — which can fit in a few hundred lines with no dependencies beyond a matrix library.
The asymmetry is the point. The algorithm is small and fully understood; anyone can read it in an afternoon. The parameters are enormous and understood by nobody. All the difficulty, all the capability, and all the mystery is on the data side of the line, not the code side.
A useful framing for what those parameters contain: a lossy compression of the internet. Fifteen trillion tokens do not fit into 140 GB, so the model cannot have stored the corpus. It stored the patterns, keeping what recurs and discarding what does not. That is exactly why it recalls the shape of a fact confidently while getting a specific date wrong — the same way you remember the plot of a film but not its dialogue.
Post-Training
Post-training converts a document simulator into an assistant. It is dramatically cheaper than pre-training — days rather than months, thousands of GPUs' worth of work rather than tens of thousands — and it changes the model's behaviour out of all proportion to its cost.
It runs in two stages that answer two different questions. Supervised fine-tuning teaches the model what to say. Reinforcement learning from human feedback teaches it how well to say it.
Supervised fine-tuning (SFT)
SFT is the same next-token objective as pre-training, pointed at a different corpus: a curated set of ideal conversations. Human annotators — working to detailed guidelines about helpfulness, honesty and safety — write exemplary exchanges. Modern pipelines generate most of this data with models and use humans to review and filter, which is what makes millions of conversations affordable.
Nothing about the mechanism changes. The model is still imitating documents. The documents are simply now transcripts of a helpful assistant behaving well, so imitating them means behaving that way. In a real sense the assistant you talk to is a statistical simulation of the human labelers who wrote the training conversations, and their instructions, blind spots, and house style are baked into it.
The chat token format
The model still only sees a flat token stream, so conversation structure has to be encoded into it. Special tokens — reserved IDs that never appear in ordinary text — mark turn boundaries and speaker roles. A conversation is flattened into one sequence, and the model is trained to produce assistant turns and stop at the end-of-turn marker.
// a conversation, flattened into the token stream the model actually sees <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user What is the capital of France?<|im_end|> <|im_start|>assistant The capital of France is Paris.<|im_end|> ↑ generation stops here
Roles are a convention encoded in tokens, not an enforced boundary. Everything — system instructions, user text, retrieved documents, tool output — arrives as one flat sequence. The model has no reliable way to tell trusted instructions from text that merely looks like instructions. Chapter 6 returns to what that costs.
RLHF and the reward model
SFT gets you a competent assistant, but "which of these two good answers is better?" is a judgment humans make easily and write down badly. RLHF exploits that asymmetry. Rather than asking annotators to author perfect responses, it asks them only to rank responses the model already produced.
- Collect preferences. Sample several responses to the same prompt; a human picks which is better.
- Train a reward model. A separate network learns to predict those human judgments, producing a scalar score for any response. It is an automated, imperfect stand-in for a human rater.
- Optimize against it. The language model is updated by reinforcement learning to produce responses the reward model scores highly, with a penalty for straying too far from the SFT model — otherwise it drifts into degenerate text that games the scorer.
RLHF is why assistants format answers well, hedge appropriately, decline gracefully, and admit uncertainty. It is also the stage most responsible for sycophancy: if raters reward agreeable answers, the reward model learns agreeableness, and the policy learns to supply it. The reward model is a proxy, and optimizing hard against any proxy eventually finds its flaws instead of your intent.
LLM Psychology
Language models fail in characteristic, repeatable ways. Each of these quirks is a direct consequence of something in Chapter 1 — none of them are mysterious once you know where they come from, and knowing where they come from is what lets you predict them instead of being surprised by them.
One warning before the list. These cards use words like "memory" and "knows" because the alternatives are unreadable. The vocabulary is borrowed, and the resemblance to human cognition is superficial. Anthropomorphizing here is the single most reliable way to form wrong predictions about what a model will do.
Hallucination
Training text is overwhelmingly written by people who knew their subject, so the model learned that confident assertion is how text about facts sounds. It has no internal signal separating recall from invention.
Mitigation: train on explicit "I don't know" examples; ground answers in retrieved sources; let the model use tools instead of recalling.
Two kinds of memory
Parametric memory is baked into the weights: vast, lossy, undated, and unverifiable. Context memory is the tokens in the current window: precise, fully attended, and gone when the conversation ends.
Practical rule: if it must be exactly right, put it in the context. Do not trust the weights to have kept it.
Tool use
Post-training teaches the model to emit specially formatted tokens that a surrounding program intercepts — a search, a calculator, a code run. Results are inserted back into the context, moving a fact from unreliable parametric memory into reliable working memory.
Consequence: the "model" you use is a system. The network is one component of it.
No persistent self
Weights are frozen after training. Nothing you say changes them. Each conversation starts from the same state, and apparent continuity across sessions is a product feature — stored text replayed into the context — not a property of the model.
Consequence: "remembering" you is retrieval plus prompt engineering, all the way down.
Stochastic by construction
Output is sampled, not computed. The same prompt yields different answers because a different token was drawn from the same distribution. Karpathy's framing is apt: a hundred-thousand-sided weighted die, rolled once per token.
Consequence: one good answer is weak evidence. Sample repeatedly before concluding a model can do something.
Knowledge cutoff
The weights stop at a date, and the model has no perception of that boundary from the inside. Asked about events past it, it does not detect a gap — it generates the most plausible continuation, which is a confident, invented answer.
Mitigation: retrieval and search tools. See Chapter 5.
The reversal curse
Facts are stored directionally. A model reliably told that A is B's parent may fail when asked who B's child is. Documented systematically in 2023 with the example of Tom Cruise and his mother, Mary Lee Pfeiffer — recognized in one direction, not the other.
Cause: next-token prediction over text where one ordering was far more common than the other.
Fixed compute per token
Every token costs one forward pass — the same arithmetic whether the next word is "the" or the answer to a hard problem. There is no mechanism to stop and think longer about a difficult step.
Workaround: chain of thought turns the output itself into a scratchpad. Emitting intermediate steps buys more forward passes, and each written step becomes context the later steps can attend to. Reasoning models make this behaviour automatic — but the chain is a scratchpad, not introspection, and does not reliably describe why the answer came out as it did.
Whether these limits are permanent is an open argument. Yann LeCun has argued that text-only training cannot reach human-level intelligence and that progress needs world models learned from sensory and physical data, with persistent memory and hierarchical planning. Others hold that scaling plus better post-training keeps delivering. This page describes the mechanism; it does not settle the forecast.
Retrieval-Augmented Generation
Chapter 4 established two facts that combine into a strategy: parametric memory is unreliable, and context memory is not. RAG is the systematic exploitation of that gap. Fetch the relevant text first, put it in the context, then ask the question.
Nothing is retrained. The weights never change. RAG shifts the model's next-token distribution by changing what it is conditioning on — moving it from "what does the internet usually say about this?" to "what does the document directly above say?"
What’s the maximum number of devices the Aegis Photon Router X4 supports on a single VLAN, and which firmware version fixed the multicast bug?
A RAG system is only as good as its retriever. Surface the wrong passage and the model will ground itself confidently in the wrong passage — a new failure mode layered on the old one, now wearing a citation. Retrieval also does nothing about the deeper issue: no part of embedding, attention, decoding or sampling contains a step that checks whether something is true before continuing. RAG improves the odds. It does not change the mechanism.
Security
Every attack below exploits the same structural fact: a language model receives one undifferentiated stream of tokens and has no reliable way to distinguish instructions it should follow from text it should merely read. Safety behaviour is a learned tendency, not an enforced boundary — and learned tendencies can be argued out of.
Jailbreaks
Wrap a forbidden request in a frame the safety training does not cover — a fictional roleplay, a hypothetical, an encoding like base64, a low-resource language, or an adversarial suffix found by automated search. The request survives; the refusal trigger does not fire.
Root cause: refusal is a learned behaviour over a distribution of phrasings, so novel phrasings fall outside it.
Prompt injection
Hide instructions in content the model will read — white text on a white background, a comment in a code file, a line buried in a retrieved document or an email. The model treats them as instructions because it cannot tell they came from an untrusted source.
Why it is getting worse: agents with tool access turn injected text into real actions on real systems.
Data poisoning & backdoors
Plant crafted text where it will be scraped into a future training corpus, pairing a rare trigger phrase with a target behaviour. The model behaves normally until the trigger appears, at which point the backdoor fires.
Why it is hard to catch: evaluation without the trigger shows a perfectly healthy model.
Adversarial inputs
Perturbations optimized against the model rather than written for a human — token sequences that read as gibberish, or noise patterns in an image that are invisible to the eye. Both can reliably steer a model's output.
Consequence: multimodal input widens the attack surface considerably.
Defenses exist — adversarial training, input and output classifiers, privilege separation between trusted instructions and untrusted content, capability limits on agents. All of them raise the cost of an attack; none of them close the category. As long as instructions and data share one channel, the attacker can search for a phrasing that works, and search is cheap. Treat model output as untrusted input to whatever consumes it.
The Full Pipeline
End to end: from a crawl of the open web to something you can hold a conversation with. Note where the cost sits — one stage consumes almost the entire budget, and the stages that determine how the model actually behaves toward you are among the cheapest.
| Stage | Input | Output | Scale / cost |
|---|---|---|---|
| 1 · Data collection | The open web | Filtered corpus | ~2.7B pages → 44 TB |
| 2 · Tokenization | Filtered text | Token IDs | ~15T tokens · 100K vocab |
| 3 · Pre-training | Token stream | Base model weights | months · ~1025 FLOPs |
| 4 · Base model | Any prefix | Plausible continuation | e.g. 140 GB @ 70B fp16 |
| 5 · SFT | Ideal conversations | Assistant behaviour | days · millions of dialogues |
| 6 · RLHF | Human rankings | Aligned preferences | days · reward model + RL |
| 7 · Deployment | Weights + tools + retrieval | The assistant | ongoing inference |
Filter the web down to a few tens of terabytes of decent text. Chop it into subword tokens. Train a stack of masked self-attention and feed-forward blocks to predict the next token, until the loss stops falling. You now have a lossy compression of the internet that can continue any document. Fine-tune it on exemplary conversations so it behaves like an assistant, then tune it against human preference rankings so it does so gracefully. Bolt on tools and retrieval so it can consult reality instead of its own weights. Sample from its output distribution one token at a time. That is the entire machine — and everything it does well, and everything it gets wrong, follows from those steps.