In this liveProject, you’ll take on the role of a full-stack developer working for a school district. Your district wants to further student learning by developing its own large language model (LLM) applications to assist students, parents, and teachers. Your goal is to develop an MVP of this by creating a chatbot that can answer questions and engage in follow-up conversations. You’ll utilize the open source Llama LLM from Meta AI to do this. Your model will need to run with very low resources—it needs to work on the school’s outdated computers—and be equipped with an intuitive user interface. Let’s get started!

In order for the Llama-based chatbot app to answer chemistry questions reliably and without hallucination, you need to ground and supplement it with facts and knowledge. That’s why your local school district has tasked you with using RAG (Retrieval Augmented Generation) to help improve the capabilities of the chemistry chatbot app. RAG allows an LLM to search a knowledge base to help answer questions, avoiding unfortunate hallucinations. You’ll create a vector database for chemistry textbooks as the knowledge base, establish an RAG API server for the chatbot, and then deploy your new bot to both the web and Discord.

Your local school district has an LLM chatbot specialized in chemistry. But it sometimes answers questions entirely unrelated to chemistry! To improve the safety of this LLM application, you are asked to come up with a new LLM that would classify student questions and return machine-readable (JSON) messages to the application frontend so that the application can decide whether to answer this question. To accomplish that, you need fine-tuning to "teach" the model how to classify questions through many examples, which themselves are generated by a Llama-based LLM, and to always respond with JSON.