7 Creating LLM-based Applications using LangChain and LlamaIndex
This chapter shows how to turn large language models into practical applications by combining LangChain for orchestration and LlamaIndex for connecting models to private data. It begins with a concise view of LLMs—their scale, pretraining and fine-tuning—and motivates moving beyond single-model pipelines toward modular systems that mix providers and capabilities. The overarching goal is to build chat and task-focused tools that can reason over both public knowledge and domain-specific documents, while balancing speed, cost, quality, and data privacy, whether models run in the cloud or locally.
With LangChain, you compose applications from prompt templates, LLMs, and output parsers into executable chains using a simple pipeline syntax. The chapter demonstrates using Hugging Face–hosted models (for example, Zephyr) via HuggingFaceEndpoint and OpenAI models via ChatOpenAI, then shows how to run the chain, parse responses, and maintain conversational context. Conversation memory is handled either by injecting history into the prompt or by using RunnableWithMessageHistory for cleaner state management. Beyond Q&A, the same pattern supports text completion and swapping in other instruction-tuned models (such as Falcon), while noting that provider-hosted inference happens remotely and local inference remains possible for select tasks.
LlamaIndex brings Retrieval-Augmented Generation to your own files by ingesting documents, creating vector embeddings, and indexing them for efficient retrieval. The workflow covers loading PDFs with SimpleDirectoryReader, embedding locally with a Hugging Face model (such as BAAI/bge-small-en-v1.5) to preserve privacy, building and persisting a VectorStoreIndex, and then querying it with either a local LLM (for example, Llama 3.2–3B Instruct) or a cloud model like gpt-4o-mini, highlighting the trade-off between speed/quality and data confidentiality. The chapter concludes by wrapping the query or chat engine in a lightweight Gradio interface, switching from as_query_engine to as_chat_engine for multi-turn conversations, and optionally accelerating local inference on available GPUs.
Large Language Models are trained using large amount of data – books, articles, web pages, videos, and images.
The input variable is linked to the variable in the string template
Creating a Hugging Face token
The 'HuggingFaceH4/zephyr-7b-alpha' is a Text Generation model.
User can ask questions and get a response from the LLM
You can get the model to complete your sentence
Applying for an OpenAI API key
The “Training Documents” folder contains four PDF files
The Gradio interface is bound to the query engine
Asking a question regarding the purchases stored in the PDF receipts
Asking a series of follow-up questions and the chatbot can maintain the conversation
The Gradio chat user interface
Using the chat user interface to chat with the chat engine
Summary
- A Large Language Model (LLM) is a type of AI model that is designed to understand and generate human-like text based on the patterns and structures it has learned from massive amount of training data.
- A "token" is a chunk of text that a model processes as a single unit.
- LangChain is a framework built around LLMs , designed to simplify the creation of applications using LLMs.
- A LangChain application is made up of components chained together.
- You can run an LLM locally or use one that is cloud-based, such as those from OpenAI.
- LlamaIndex is a data framework for LLM-based applications to ingest, structure, and access private or domain-specific data.
- RAG is a technique used to enhance the performance of large language models (LLMs) by integrating them with an external retrieval system.
- Running a model locally for vector embedding ensures your data stays within your computer.
- You can create a web frontend for your LLM applications using Gradio
Hugging Face in Action ebook for free
