Public LLMs from vendors like OpenAI and Anthropic are impressive, but using them isn’t an option when you’re dealing with sensitive corporate information, regulated data, or private personal information (PPI). And for high-volume applications, costs quickly soar as usage grows. Self-Hosting LLMs at Scale, by Hugging Face platform engineer Arek Borucki, shows you how to move those workloads onto infrastructure your organization controls, using vLLM as the inference engine and Kubernetes as the platform. You’ll learn how to turn an open-weight model into a reliable internal service that your developers can call just like any commercial API.
Whether you’re bringing you LLM in-house for privacy, cost, compliance, or control, this book gives you the arithmetic, the engineering, and the operational discipline to do it well. It lays out the engineering path you need to serve open-weight models privately, reliably, and at a cost you can defend. Arek Borucki shows you how to apply what you already know about traditional platform engineering to serving LLMs as you transition a law firm from insecure public chatbots to a company-wide AI endpoint.
Every number in the book comes from a recorded lab session, and the early labs run on a free notebook GPU so you can reproduce the results without buying hardware. You’ll calculate GPU memory and cost for a model and workload before anything is deployed, choose an open-weight model using a golden set of questions built from your own work, and serve it behind an OpenAI-compatible API with streaming and sampling under your control.
To give you confiodenc—and proof—that your efforts are delivering positive results, you’ll benchmark the endpoint to find its real capacity limit, quantize larger models, and split one model across several GPUs or servers when it outgrows a single card. By the final chapters, you’ll run inference on Kubernetes with a gateway, quotas, monitoring, and safe rollouts.
Finally, you’ll see how to establish a model selection process that survives both staff turnover and new model releases, and explore vLLM internals that let you effectively tune a running system against real service level objectives. By the end, you’ll be operating a fully functional company platform, with model traffic routing, quota enforcing, full governance records, and even the ability to redirect to a commercial model via API when you need to.
MongoDB 8.0 in Action, Third Edition you’ll learn how to take advantage of all the new features of MongoDB 8.0, including the powerful MongoDB Atlas multi-cloud data platform. You’ll start with the basics of setting up and managing a document database. Then, you’ll learn how to use MongoDB for AI-driven applications, implement advanced stream processing, and optimize performance with improved indexing and query handling. Hands-on projects like creating a RAG-based chatbot and building an aggregation pipeline mean you’ll really put MongoDB into action!