Build Your LLM and Fine-Tune it for Real Tasks

In this liveProject, you’ll build a whole large language model from the ground up! You’ll step into the role of a team member at X Education, tasked by your managers with understanding how AI works by creating one of your own. Following the open source Llama 3.2 architecture, you’ll start by coding modular architecture components and assembling them into a GPU-ready model. You’ll implement a tokenizer that seamlessly integrates with the system, fine-tune the model on a sample dataset, and load pre-trained weights to unlock full functionality. Finally, you’ll generate, structure, and refine text outputs in a chat format, until you have a polished, user-friendly model running locally on your hardware.

X Education gets thousands of potential leads every day—so why do the sales team keep chasing cold leads instead of promising prospects? In this liveProject, your boss has tasked you to solve this problem by creating a powerful lead-scoring model using LLM technology. You’ll take a base purpose model and train it on X Education’s data to become a focused machine lead evaluating machine! Begin by setting up a resource-efficient environment and load the pre-trained model, then transform X Education’s 9,000-lead dataset into tokenized prompts. You’ll fine-tune Llama 3.2 to classify leads for conversion potential, and finally evaluate its performance against a few-shot baseline.

AI will be based on a pretrained Llama 3.2 model and be entirely based on-site with no reliance on external services. You’ll set up your Colab environment with Unsloth and preprocess a commit-style dataset into instruction–input–output triples. Next, benchmark the base model’s code generation skills, then fine-tune it with LoRA. Finally, you’ll evaluate the fine-tuned model against the baseline, comparing consistency, correctness, and output quality.