Appendix E Parameter-efficient Finetuning with LoRA

This appendix introduces low-rank adaptation (LoRA) as a practical, parameter-efficient way to finetune large language models on task-specific data. Instead of updating all pretrained weights, LoRA learns a compact, low-dimensional approximation of the weight updates, which captures the most influential directions of change while keeping compute and storage demands low. Because the learned adapters are kept separate from the original model, one can customize a single base model for multiple tasks or clients by storing only the small LoRA weights, and the approach applies to both classification-style finetuning and supervised instruction finetuning.

Conceptually, LoRA replaces direct weight updates with the product of two small matrices and combines their contribution with the frozen pretrained weights during the forward pass. A pair of hyperparameters controls capacity and influence: rank determines the adapter size (and thus trainable parameter count), while alpha scales the adapter’s contribution. In practice, the appendix shows how to implement a LoRA layer, wrap existing Linear layers with a Linear-with-LoRA module, and recursively swap these throughout the transformer’s attention, feed-forward, and output layers. After freezing the original GPT-2 weights, only the LoRA parameters are trained; with rank=16 and alpha=16, this reduces trainable parameters from about 124 million to roughly 2.7 million, and the initialization preserves original behavior until training begins.

The appendix then reprises the chapter 6 workflow on SMS spam classification: preparing the dataset, loading a pretrained GPT-2 small model, replacing the output head for classification, and training only the LoRA adapters. Although LoRA adds a bit of forward-pass overhead, the savings during backpropagation make it especially attractive for larger models. The resulting model converges quickly and achieves very high performance—perfect training accuracy and around 97–98% on validation and test—demonstrating strong task adaptation with a tiny fraction of the original parameters. Overall, LoRA delivers a compelling balance of efficiency, scalability, and effectiveness for finetuning large language models.

Figure E.1 A comparison between weight update methods: regular finetuning and LoRA. On the left, regular finetuning involves updating the pretrained weight matrix W directly with ΔW. On the right, LoRA uses two smaller matrices A and B to approximate ΔW, where the product AB is added to W, and r denotes the inner dimension, a tunable hyperparameter.

Figure E.2 The LoRA matrices A and B are applied to the layer inputs and are involved to compute the model outputs. The inner dimension r of these matrices serves as a setting that adjusts the number of trainable parameters by varying the sizes of A and B.

Figure E.3 Illustration of the integration of LoRA into a model layer. The original pretrained weights (W) of a layer are combined with the outputs from LoRA matrices (A and B), which approximate the weight update matrix (∆W). The final output is calculated by adding the output of the adapted layer (using LoRA weights) to the original output.

Figure E.4 A diagram representing the architecture of the GPT model. It highlights the parts of the model where Linear layers are being upgraded to LinearWithLoRA layers for parameter-efficient finetuning.

Figure E.5 The training and validation loss curves over six epochs for a machine learning model. Initially, both training and validation loss decrease sharply, then they level off, indicating the model is converging, which means that it is not expected to improve noticeably with further training.

FAQ