2 LLM Basics

The chapter introduces large language models as systems that can be used effectively even without understanding their internals, but argues that a basic grasp of how they work becomes invaluable when outputs go wrong. At their core, LLMs predict the next token one step at a time, building long responses through repeated probability-based continuations rather than by planning an entire answer in advance. This simple loop, trained on vast amounts of text, explains how models can generate coherent language, code, and explanations that often feel surprisingly capable.

It then explains the main building blocks that turn raw text into model-ready input. Text is first split into tokens, which may be full words or smaller pieces, and those tokens are converted into numerical IDs. Because IDs alone carry no meaning, embeddings transform them into high-dimensional vectors that place related tokens near one another in semantic space. Attention then lets each token weigh the relevance of other tokens in the same input, allowing the model to resolve references, connect distant ideas, and produce context-aware outputs. The chapter also emphasizes the limits of the context window: the model can only “see” the tokens currently inside it, and important details can be lost if they fall outside that range or even fade in long inputs.

Finally, the chapter shows how a pretrained language model becomes an assistant. Pretraining teaches it the statistical structure of language, but not how to answer users helpfully. Fine-tuning adds instruction-following behavior using curated prompt-response examples, and alignment through human feedback shapes the model toward responses people judge as useful, safe, and clear. These learned behaviors create guardrails, which are not hard-coded rules but patterns that emerge from training. The chapter closes by noting that specialized models build on the same foundation and adapt it to domains like code, law, medicine, or multimodal inputs, all while relying on the same underlying mechanism: predicting what comes next.

The same word, seen in two ways. A human reads "unbelievable" as a single unit of meaning. A model never sees the word at all: it receives three tokens and three numbers.

Tokens that appear in similar contexts end up close to each other in the embedding space. The three tokens of "unbelievable" each land in a different neighborhood: "un" near other negating prefixes, "believ" near words of trust and certainty, "able" near other adjectival suffixes. The model never defined these clusters; they emerged from patterns in the training data.

When processing "it," the model assigns an attention weight to every other token in the sentence. "Performance" receives the highest weight by far (0.68 out of 1.0)which is how the model infers what "it" refers to.

The context window defines the boundary of what the model can see. Tokens inside it are fully available to the attention mechanism. Tokens outside it are gone. In a long conversation, early messages can silently fall outside the window, taking their context with them.

Even within the context window, not all tokens receive equal attention. Research shows that models tend to attend more strongly to information at the beginning and end of a long input, while content in the middle may be partially overlooked. This means that where you place information matters, not just whether it fits inside the window.

A child and a language model learn language through the same fundamental mechanism: exposure and pattern recognition, without explicit instruction.

The same prompt, fed to two different models. The foundation model treats it as text to continue; it doesn't recognize an instruction, it simply extends the pattern. The fine-tuned conversational model reads it as a request and responds accordingly. The input is identical. What changes is how the model has learned to interpret it.

The RLHF loop applied to a single prompt. The model generates two candidate responses; human reviewers rank them by helpfulness and clarity; a reward model learns that preference; the LLM is updated to make better responses more likely. The process repeats until the model's behavior aligns with human expectations.

The four stages that turn a prediction engine into a specialist. Pretraining builds a foundation model from trillions of tokens; fine-tuning teaches it to follow instructions; alignment shapes its behavior through human feedback. Specialization is the final step: the same foundation, redirected toward a specific domain. The core mechanism never changes: predict the next token, one step at a time.

• At their core, LLMs are prediction engines: given a sequence of tokens, they estimate the probability of the next token and generate responses one token at a time.
• LLMs work with tokens, small pieces of text produced by algorithms. Each token is converted into a numerical ID, which is then transformed into an embedding: a rich vector representation that encodes relationships between tokens based on how they appear in context.
• Attention is the mechanism that allows each token to consider every other token in the sequence and determine their relevance. This is what enables LLMs to resolve references, maintain coherence, and handle complex relationships across long passages without processing text strictly left-to-right.
• The context window defines the boundary of what the model can see at any given moment. Everything inside it is available to the attention mechanism. Everything outside it does not exist, as far as the model is concerned. In long conversations, early content can silently fall outside the window because it is no longer visible.
• Pretraining is the phase in which a model is exposed to trillions of tokens and learns the statistical patterns of language. No rules are written by hand. Grammar, tone, style, and reasoning-like behavior all emerge indirectly through the process of predicting what comes next in the same way a child absorbs language through exposure.
• Fine-tuning transforms a foundation model into a conversational assistant by training it on carefully curated prompt-response pairs. Before fine-tuning, the model treats the input as text to continue. After fine-tuning, it interprets input as an instruction to follow. That distinction is subtle but fundamental.
• Alignment training shapes how the model behaves toward humans. Human reviewers rank model-generated responses; those rankings train a reward model, which then guides the LLM's updates. The result is a model that has learned what responses humans tend to prefer.
• Alignment produces guardrails: learned behaviors that define what the model will and won't do. Unlike hard-coded rules, guardrails are more flexible than a blocklist, but also imperfect: they reflect the priorities and blind spots of the people who provided the feedback, vary between models, and can sometimes be bypassed by inputs the training process never anticipated.
• Specialized models extend this arc further. Starting from the same pretrained foundation, additional training on domain-specific data produces models optimized for coding, medicine, law, or multimodal tasks. The core mechanism of predicting the next token never changes, but what the model has learned to do with it keeps growing.

1. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language models are few-shot learners. Proceedings of the 34th International Conference on Neural Information Processing Systems. Curran Associates Inc.
2. Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2023). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173.
3. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Proceedings of the 27th International Conference on Neural Information Processing Systems, 2, 3111–3119.
4. Sennrich, R., Haddow, B., & Birch, A. (2016). Neural machine translation of rare words with subword units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1715–1725. https://doi.org/10.18653/v1/P16-1162
5. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, 6000–6010. Curran Associates Inc.

FAQ