Build Your First LLM from Scratch

you own this product
prerequisites
Intermediate Python • Beginner PyTorch • Basic ML concepts
skills learned
Implementing Llama 3.2 architecture with Python and PyTorch • Configuring GPU compatibility • Training on toy datasets and loading Hugging Face weights
Tobias Zwingmann
Included with a Manning Online subscription

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

Look inside

In this liveProject, you’ll build a whole large language model from the ground up! You’ll step into the role of a team member at X Education, tasked by your managers with understanding how AI works by creating one of your own. Following the open source Llama 3.2 architecture, you’ll start by coding modular architecture components and assembling them into a GPU-ready model. You’ll implement a tokenizer that seamlessly integrates with the system, fine-tune the model on a sample dataset, and load pre-trained weights to unlock full functionality. Finally, you’ll generate, structure, and refine text outputs in a chat format, until you have a polished, user-friendly model running locally on your hardware.

This project is designed for learning purposes and is not a complete, production-ready application or solution.

project author

Tobias Zwingmann
Tobias Zwingmann is a leading AI expert and advisor. As Managing Partner at Germany-based RAPYD.AI, he helps companies adopt AI and machine learning faster while achieving meaningful business impact. Leveraging his background as a data scientist and over 15 years of experience at the intersection of business and technology, Tobias has led AI initiatives across a wide range of industries, from early-stage prototyping to full-scale deployment.

prerequisites

This liveProject is for intermediate Python programmers who understand both data processing and machine learning basics, and are keen to adapt these skills to creating and tuning Large Language Models.


TOOLS
  • Intermediate Python
  • Basics of Google Colab
  • Basics of Pandas
  • Basics of PyTorch
  • Basics of Huggingface
TECHNIQUES
  • Basics of data preprocessing
  • Basics of machine learning
  • Basics of Text Data Handling

features

Self-paced
You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from fellow participants and even more help with paid sessions with our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.
choose your plan

pro

monthly
annual
$24.99
$249.99
only $20.83 per month
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Build Your First LLM from Scratch project for free

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Build Your First LLM from Scratch project for free