Three-Project Series

Open Source LLMs on Your Own Computer you own this product

Intermediate Linux • Intermediate ChatGPT • Basic Git • Basic compilers • Basic Python
skills learned
Building LLM chatbot apps • Building a RAG vector store for external knowledge • Fine-tuning an LLM
Michael Yuan and Tony Yuan
3 weeks · 4-6 hours hours per week average · BEGINNER

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases

lite $19.99 per month

  • access to all Manning books, including MEAPs!


5, 10 or 20 seats+ for your team - learn more

Your school district needs you! In this series of liveProjects, you’ll assist them by developing a custom chatbot for teaching students chemistry based on Meta AI’s Llama large language model. To do this, you’ll utilize entirely open-source tools, including llama.cpp, WasmEdge, Git, Qdrant, Python, and Hugging Face. Your model will need to run with very low resources—it needs to work on the school’s outdated computers—and be equipped with an intuitive user interface. Once you’ve developed your foundation, you’ll fine-tune it for knowledge domains and establish a RAG supplemental knowledge base to ensure it can’t hallucinate!

These projects are designed for learning purposes and are not complete, production-ready applications or solutions.

I was impressed with the instructions and how clear they were, especially as the topic is complex.

Chris Parsonson, Agile ICT, Helpdesk Manager

here's what's included

Project 1 Chatbot with Llama

In this liveProject, you’ll take on the role of a full-stack developer working for a school district. Your district wants to further student learning by developing its own custom large language model (LLM) to assist students, parents, and teachers. Your goal is to develop an MVP of this by creating a chatbot that can answer chemistry questions and provide follow-up answers and conversations. You’ll utilize the open source Llama LLM from Meta AI to do this. Your model will need to run with very low resources—it needs to work on the school’s outdated computers—and be equipped with an intuitive user interface. Let’s get started!

Project 2 Add Knowledge to the Chatbot

Fine-tuning an LLM is time-consuming and expensive! That’s why your local school district has tasked you with using RAG (Retrieval Augmented Generation) to help improve the capabilities of a chemistry chatbot based on Meta AI’s Laama. RAG allows an LLM to search a database to help answer questions, avoiding any unfortunate hallucinations. You’ll create a vector database for external knowledge for your chatbot’s RAG, establish an RAG API server for it to use, and then deploy your new bot to both the web and Discord.

Project 3 Fine-Tune the Llama Model

Your local school district has a basic LLM chatbot based on Llama—now it needs fine-tuning! That’s where you come in. In this liveProject, you’ll utilize the Supervised Fine Tuning (SFT) approach to customize a chemistry chatbot for teaching children. SFT uses question-and-answer pairs to train a model to answer questions with a given answer. To achieve this, you will need to prepare your training data, utilizellama.cpp tools for LORA fine-tuning, and CLI tools to test inference on a fine-tuned model.

book resources

When you start each of the projects in this series, you'll get full access to the following book for 90 days.

choose your plan


only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • Open Source LLMs on Your Own Computer project for free

I felt the progression laid out by the series was a good set of stepping stones through the subject material

Brandon Hunt, Software Architect, Intrado

project authors

Michael Yuan

Dr. Michael Yuan is the founder and maintainer of the open -source WasmEdge project — one of the most popular WebAssembly (Wasm) runtimes for server and edge applications. He is also the creator of LlamaEdge, a cross-platform, embeddable, and cloud-native runtime for AI applications, supporting LLMs and popular vision and speech models. Dr. Yuan is the author of 5 books on software engineering and a frequent speaker at conferences such as KubeCon, Open Source Summit, QCon, Rust Conf, VMWare Explore, COSCon, and Wasm I/O. He has designed and implemented enterprise applications and server-side software infrastructure since the early 2000s. Dr. Yuan holds a PhD in Astrophysics from the University of Texas at Austin.

Tony Yuan
Tony Yuan is a student at Kealing Middle School in Austin, Texas, who loves technology and AI. He served as a TA for his school’s computer science class and as the leader of Kealing’s philharmonic orchestra. He enjoys listening to rock music and reading history books when he's not programming or studying.


These liveProjects are for software developers and IT professionals who are interested in building LLM applications in their own domains. To begin, you will need to know the following:

  • Intermediate Linux
  • Basics CMAKE and Linux dev tools
  • Basics of Git
  • Basics of Hugging Face
  • Basics of Python

  • Command line tools
  • Basics of LLMs
  • Basics of model training and inference
  • Deploying and testing web services


You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from other participants and our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.