11 Large Language Models (LLMs)

Large language models extend the simple idea of next‑word prediction into systems that learn rich patterns across massive corpora using billions of parameters and the Transformer architecture. Built on attention, deep neural networks, and vast, carefully curated text (and sometimes code), they move far beyond n‑grams to generate coherent prose, answer questions, translate, and reason in context. The chapter builds intuition by contrasting phone‑style autocomplete with modern LLMs, then situates LLMs in the broader ML landscape, emphasizing why scale—of tokens, parameters, and data diversity and quality (including licensing and safety)—matters for generalization and capability.

The workflow begins with preparing data: language filtering, boilerplate and near‑duplicate removal, normalization, and safety/PII scrubbing. Text is tokenized (e.g., via byte‑pair encoding) to form a vocabulary and token IDs, then streamed into fixed‑size context windows. Tokens are mapped to trainable embeddings and augmented with positional encodings so order is preserved. A Transformer block applies self‑attention (Query–Key–Value projections, scaled dot‑product, softmax weights), often across multiple heads, followed by residual connections, layer normalization, and a feed‑forward network; stacking many blocks yields progressively richer representations. Decoding projects final vectors to vocabulary logits, converts them to probabilities, and computes loss (cross‑entropy) against the true next token; gradients backpropagate through all components, and training iterates over batches and epochs, typically accelerated on GPUs.

Controlling and improving LLMs involves managing epochs and checkpoints, early stopping on validation loss to avoid overfitting, and tuning hyperparameters (learning rate, batch/context size, width/depth, attention heads). At inference time, sampling controls like temperature balance determinism and creativity; LLMs excel at zero‑shot and few‑shot in‑context learning. Beyond pretraining, Reinforcement Learning from Human Feedback aligns models with human preferences; Mixture‑of‑Experts scales capacity efficiently by routing tokens to specialized subnets; and Retrieval‑Augmented Generation grounds outputs in external, up‑to‑date sources. Together, these techniques power practical applications—content generation, information synthesis and summarization, coding assistance, and product experiences that turn unstructured inputs into organized, actionable outcomes.

Example of autocomplete as a language model

How generative models and Transformers fit into machine learning

How we subconsciously identify patterns in language

Formula for probability of a word occurring after another word

Example of the probability of words occurring

Comparison of size of training sets in popular language models

An overview of the LLM training workflow

Preparing training data step in the LLM training workflow

Sample training datasets and their domain composition

Excerpt of training data from Alice’s Adventures in Wonderland

Training data with boiler-plate stipped

Training data normalized

Encoding step in the LLM training workflow

Training data split into characters with end-of-word markers

Training data after one iteration of merging with BPE

Training data after two iterations of merging with BPE

Training data after three iterations of merging with BPE

Original training data

Tokenized training data as a token stream

Tokenized training data as training batches

Designing the architecture step in the LLM training workflow

Comparison of different ANN architectures aimed at training LLMs

The fundamental composition of an artificial neural network

A simplistic overview of the Transformer framework

The data shapes for the different components in our Transformer framework

Continuation of the encoding step in the LLM training workflow

Sampling the first batch of tokens from the training data

A view of how the vocabulary size and width dimensions shape the embedding

An odometer analogy for how sinusoidal positional encoding works

A breakdown of the calculations involved in sinusoidal encoding

A view of the embedding combination step in the encoding stage

Where self-attention fits into the Transformer framework

An analogy of the purposes of Query, Key, and Value

Initialization of the values for Queries, Keys, and Values

An analogy of how self-attention works

The formula for scaling scores given the score and dimensions

How softmax scales scores to probabilities

A subset of the context vector calculations

An analogy of the usefulness of multiple attention heads

Calculations for the residual connection

The formula for layer normalization

The decoding step in the LLM training workflow

The architecture of the feed-forward network to output token probabilities

Calculations for feed-forward outputs

Calculations for adding bias to the output

Calculations for applying the ReLU activation function

Calculations for the project-down layer output

Calculations for adding bias to the output of the project-down layer

Calculations for adding the residual

How multiple transformer blocks are stacked

Calculations for logits for tokens

Final prediction for the next token after “a”

Revisiting the first training batch of tokens

Calculations for new weights given the learning rate and gradient

How epochs repeat in the Transformer framework

Example of use cases for few-shot learning

FAQ