click to
look inside
Look inside
Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
You can see any available part of this book for free.
Click the table of contents to start reading.
ASK me anything...
we'll search our titles
to answer your question

Tuning Up you own this product

From A/B testing to Bayesian optimization
David Sweet
  • MEAP began December 2020
  • Publication in Fall 2022 (estimated)
  • ISBN 9781617298158
  • 275 pages (estimated)
  • printed in black & white
filed under

placing your order...

Don't refresh or navigate away from the page.
eBook Our eBooks come in DRM-free Kindle, ePub, and PDF formats + liveBook, our enhanced eBook format accessible from any web browser. $24.99 $39.99 you save $15 (38%)
Tuning Up (eBook) added to cart
continue shopping
go to cart

print book Receive a print copy shipped to your door + the eBook in Kindle, ePub, & PDF formats + liveBook, our enhanced eBook format accessible from any web browser. $34.99 $49.99 you save $15 (30%)
FREE domestic shipping on orders of three or more print books
Tuning Up (print book + eBook) added to cart
continue shopping
go to cart

This book is the sure way ticket to help you learn in an interesting way using practical code, probability, and real world examples to design and test your experiments.

Satej Kumar Sahu
Look inside
Learn practical and modern experimental methods used by engineers in technology and trading.

In Tuning Up: From A/B testing to Bayesian optimization you will learn how to:

  • Design, run, and analyze an A/B test
  • Break the "feedback loops" cause by periodic retraining of ML models
  • Increase experimentation rate with multi-armed bandits
  • Tune multiple parameters experimentally with Bayesian optimization
  • Clearly define business metrics used for decision making
  • Identify and avoid the common pitfalls of experimentation

Tuning Up: From A/B testing to Bayesian optimization is a toolbox for optimizing machine learning systems, quantitative trading strategies, and more. You’ll start with a deep dive into tests like A/B testing, and then graduate to advanced techniques used to measure performance in highly competitive industries like finance and social media. The tests in this unique, practical guide will quickly reveal which approaches and features deliver real results for your business.

about the technology

Tuning your systems is best done by following established methods employed by high-performing teams like the ones led by author David Sweet. This book reveals experiments, tests, metrics, and industry-tested tools that will ensure your projects are constantly improving, delivering revenue, and ensuring user satisfaction.

about the book

Tuning Up: From A/B testing to Bayesian optimization teaches you reliable techniques for evaluating new features and fine-tuning parameters. You’ll learn to optimize production systems with methods that have been proven in highly competitive environments. Each method is fully explained using basic math and Python code, and illustrated with real-world use cases in quantitative trading, recommender systems, and ad serving.

You’ll learn how to evaluate the changes you make to your system and ensure that your testing doesn’t undermine revenue or other business metrics. By the time you’re done, you’ll be able to seamlessly deploy experiments in production while avoiding common pitfalls.

about the reader

For ML engineers, quantitative traders, and software engineers looking to extract the most value from their systems. Examples in Python and NumPy.

about the author

David Sweet has worked as a quantitative trader at GETCO and a machine learning engineer at Instagram, where he used experimental methods to tune trading systems and recommender systems. This book is an extension of his lectures at NYU Stern and is the basis for his course, Experimental Optimization, at Yeshiva University.

FREE domestic shipping on orders of three or more print books

I would tell my students and colleagues that if they are looking for a great conversation with a knowledgeable friend then they need to pick up a copy of this book!

Roger Le

This book is a good introduction to testing and optimizations for engineering systems and applications.


This is a must read for many SWE and MLE at all levels. I work with many people with these qualifications, and too few are aware of the problems—and solutions—exposed in the book.

Eric Platon

If you're running experiments in production to optimize metrics like CTR, this is the book for you. You'll cover a lot of ground at good depth, from A/B testing to MABs with Thompson sampling.

Matt Sarmiento

Shows a non-mathematical programmer exactly what they need to write powerful mathematically-based testing algorithms.

Patrick Goetz