Pere Martra

Inference Engineering

Find out what’s slowing your LLM down—and what you can do about it. This eBook brings together five chapters from Manning books to help you make informed choices about inference, model efficiency, and production deployment. Explore ways to ease memory pressure, balance quantization savings with output quality, and reduce data movement with fused Triton kernels. Then learn how batching, streaming, and rate limiting help you handle requests in a running service. Whether you’re refining a prototype or preparing for deployment, you’ll gain practical techniques to try on your own workloads and a stronger foundation for further study of LLM optimization and GPU programming.

Local AI and Small Language Models

Choose a model that suits your task, adapt it to your needs, and build software that runs locally. This free eBook features four chapters from four Manning books, covering small language models, specialization with LoRA and DoRA, CPU inference with GGUF and llama.cpp, and application development through Ollama’s Python library. Get practical guidance on producing structured outputs, making informed quantization choices, and handling streaming responses and chatbot conversation history. Get a firmer grasp of the choices behind local AI as well as concrete techniques to try in your next project, without depending on a hosted inference service.

Rearchitecting LLMs

By default, general purpose LLMs are not optimized for specific domains and business goals. Using techniques like specialized fine-tuning, pruning unnecessary neural components, and knowledge distillation, you can rearchitect your models to cost less, run faster, and deliver more accurate results.

Rearchitecting LLMs: Structural techniques for efficient models turns research from the latest AI papers into production-ready practices for domain-specific model optimization. As you work through this practical book, you’ll perform hands-on surgery on popular open-source models like Llama-3, Gemma, and Qwen to create cost-effective local small language models (SLMs). Along the way, you’ll learn how to combine behavioral analysis with structural modifications, identifying and removing parts that don’t contribute to your model’s goals, and even use “fair pruning” to reduce model bias at the neuron level.