This chapter explains how to adapt open source foundation models to a specialized domain, focusing mainly on data preparation, retrieval-based augmentation, full fine tuning, and parameter-efficient alternatives. It emphasizes that customization begins with structuring domain data correctly: classification datasets need labeled text, generative datasets need prompt-completion pairs, and retrieval systems need documents converted into embeddings for similarity search. Across model types such as BERT, GPT-2, DistilBERT, and FLAN-T5, the overall workflow is similar when using the Hugging Face ecosystem: tokenize inputs, normalize lengths through padding or truncation, convert tokens and labels into tensors, create datasets and data loaders, load a suitable pretrained model, and then train or adapt it for the task.
The chapter presents Retrieval Augmented Generation as an alternative to changing model weights. In this approach, relevant information is retrieved from an external knowledge base and supplied to the language model as context during generation. The text explains how embeddings can be created with sentence-transformer models and stored in a vector index such as FAISS, enabling efficient nearest-neighbor search over large corpora. RAG is especially useful when the model needs current, traceable, or frequently updated information, and it can reduce hallucinations by grounding answers in retrieved sources. However, retrieval alone may not solve cases where the model lacks the ability to perform the target task or where domain patterns must be learned more deeply.
The chapter then describes full fine tuning and parameter-efficient fine tuning. Full fine tuning updates all model parameters on a smaller, labeled, task-specific dataset, illustrated with a question-answering example using DistilBERT and the Hugging Face Trainer API. Parameter-efficient fine tuning updates only a small fraction of parameters, reducing compute and cost while preserving much of the benefit of adaptation. LoRA is highlighted as a practical PEFT method that freezes the pretrained weights and trains lightweight adapter components, demonstrated with FLAN-T5 on a dialogue summarization dataset. The chapter concludes that choosing between RAG, full fine tuning, and PEFT depends on the use case: RAG is strong for fresh and interpretable knowledge access, while fine tuning or PEFT is better when the model must learn domain-specific behavior, terminology, structures, or complex task patterns.
The typical RAG workflow.
The typical fine tuning workflow.
The LoRA fine tuning process
Summary
BERT models require tokenization with special tokens CLS and SEP to mark sequence boundaries for classification tasks.
GPT models use context-target pairs where the model learns to generate the target text given the context input.
Tokenizers convert text into numerical representations that models can process during training and inference.
FAISS creates vector indexes for efficient similarity search across large datasets of text embeddings.
RAG combines information retrieval with text generation to provide models with external knowledge sources.
RAG retrieves relevant information from vector databases before generating responses to improve accuracy.
Fine tuning adjusts all parameters of a pre-trained model using task-specific labeled datasets.
LoRA fine tunes only a small subset of model parameters while keeping the original weights frozen.
PEFT techniques like LoRA reduce computational costs compared to full fine tuning while maintaining performance.
RAG works better for up-to-date information and reduces hallucinations through source traceability.
Fine tuning produces better results for complex domain-specific tasks requiring specialized knowledge.
Vector databases store embeddings and enable fast similarity search for RAG implementations.
FAQ
What is the main goal of tuning a model for a specific domain?Domain tuning adapts a pre-trained language model to a specific task, dataset, or business context. This can be done through full fine tuning, parameter-efficient fine tuning such as LoRA, or Retrieval Augmented Generation, depending on whether the model needs to learn new behavior, use external knowledge, or access up-to-date information.How should data be prepared for BERT fine tuning?For BERT fine tuning, data should usually be organized as labeled text samples, such as pairs of text and class labels. The text must be tokenized with the model-specific tokenizer, such as BertTokenizer for bert-base-uncased. Special tokens like [CLS] and [SEP] are added, sequences are padded or truncated to a consistent length, labels are encoded numerically, and the dataset is split into training and validation sets before creating PyTorch data loaders.Why is tokenization important when preparing data for LLMs?Tokenization converts text into units that the model can process. Tokens may represent words, subwords, characters, punctuation, or special symbols. The tokenizer affects vocabulary handling, out-of-vocabulary words, input size, memory usage, processing time, and overall model performance. Since LLMs have token limits, padding and truncation are also important during data preparation.How is data preparation different for GPT fine tuning compared with BERT fine tuning?BERT is often used for tasks such as classification, where each text sample has a label. GPT-style models are commonly used for generation or completion tasks, where each example includes a prompt or context and a target completion. For GPT fine tuning, the prompt and completion are formatted together, tokenized with a GPT tokenizer, converted into token IDs, padded or truncated, and loaded through a custom dataset and data loader.What is Retrieval Augmented Generation?Retrieval Augmented Generation, or RAG, is a technique that combines information retrieval with language generation. A retrieval system first searches a knowledge base for relevant information based on the user prompt. Then the LLM uses the retrieved context to generate a more accurate and informed natural language response.How is data prepared for a RAG system?For RAG, the knowledge base is converted into embeddings, which are numerical vector representations of text. These embeddings are stored in a vector database or similarity-search index, such as FAISS. At query time, the user query is also converted into an embedding, and the system retrieves the most similar stored vectors to provide relevant context to the LLM.What is FAISS and why is it useful for RAG?FAISS, or Facebook AI Similarity Search, is an open source library from Meta Research for efficient similarity search and clustering of dense vectors. It can scale to very large datasets and supports nearest-neighbor search, including GPU acceleration through CUDA. In RAG pipelines, FAISS helps retrieve relevant documents or text chunks quickly by comparing vector embeddings, often using L2 distance.What is full fine tuning?Full fine tuning is the process of continuing the training of a pre-trained model on a smaller, task-specific dataset. During full fine tuning, the model’s weights are updated based on the new data so that the model becomes better adapted to a specific task or domain. It is similar to pre-training, but uses a smaller, labeled, task-focused dataset.What is LoRA and how does it differ from full fine tuning?LoRA, or Low-Rank Adaptation, is a parameter-efficient fine-tuning technique. Instead of updating all model parameters, LoRA keeps the original pre-trained weights frozen and trains only a small number of additional parameters. This greatly reduces computational cost while often preserving strong accuracy. In the chapter’s FLAN-T5 example, only about 0.89% of the total model parameters are trained.When should I choose RAG instead of fine tuning?RAG is often preferable when the application needs up-to-date information, access to changing private or public knowledge sources, lower training cost, better traceability, and reduced hallucination through source grounding. Fine tuning is usually better when the model must learn complex domain-specific patterns, specialized jargon, task behavior, or relationships that cannot be solved by simply adding retrieved context. There is no universal best choice; the decision depends on the use case, data, cost, infrastructure, and accuracy requirements.
pro $24.99 per month
access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
Domain-Specific Small Language Models ebook for free
