Human-in-the-Loop Machine Learning

Active learning and annotation for human-centered AI
Robert (Munro) Monarch
  • MEAP began July 2019
  • Publication in June 2021 (estimated)
  • ISBN 9781617296741
  • 424 pages (estimated)
  • printed in black & white

placing your order...

Don't refresh or navigate away from the page.
print book Receive a print copy shipped to your door + the eBook in Kindle, ePub, & PDF formats + liveBook, our enhanced eBook format accessible from any web browser. $38.99 $59.99 you save: $21 (35%) pBook + eBook + liveBook
Additional shipping charges may apply
FREE domestic shipping on orders of three or more print books
Human-in-the-Loop Machine Learning (print book) added to cart
continue shopping
go to cart

eBook Our eBooks come in Kindle, ePub, and DRM-free PDF formats + liveBook, our enhanced eBook format accessible from any web browser. $31.19 $47.99 you save: $17 (35%) 3 formats + liveBook
FREE domestic shipping on orders of three or more print books
Human-in-the-Loop Machine Learning (eBook) added to cart
continue shopping
go to cart

I've learned a lot of new things about Machine Learning I would never even have considered before.

Michiel Trimpe
Look inside
Most machine learning systems that are deployed in the world today learn from human feedback. However, most machine learning courses focus almost exclusively on the algorithms, not the human-computer interaction part of the systems. This can leave a big knowledge gap for data scientists working in real-world machine learning, where data scientists spend more time on data management than on building algorithms. Human-in-the-Loop Machine Learning is a practical guide to optimizing the entire machine learning process, including techniques for annotation, active learning, transfer learning, and using machine learning to optimize every step of the process.

about the technology

“Human-in-the-Loop machine learning” refers to the need for human interaction with machine learning systems to improve human performance, machine performance, or both. Most machine learning projects do not have the time or budget for human input on every data point, and so need strategies for deciding which data points are the most important for human review. Ongoing human involvement with the right interfaces expedites the efficient labeling of tricky or novel data that a machine can’t process, reducing the potential for data-related errors.

about the book

Human-in-the-Loop Machine Learning is a guide to optimizing the human and machine parts of your machine learning systems, to ensure that your data and models are correct, relevant, and cost-effective. 20-year machine learning veteran Robert (Munro) Monarch lays out strategies to get machines and humans working together efficiently, including building reliable user interfaces for data annotation, Active Learning strategies to sample for human feedback, and Transfer Learning. By the time you’re done, you’ll be able to design machine learning systems that automatically select the right data for humans to review and ensure that those annotations are accurate and useful.

what's inside

  • Active Learning to sample the right data for humans to annotate
  • Annotation strategies to provide the optimal interface for human feedback
  • Techniques to select the right people to annotate data and ensure quality control
  • Supervised machine learning design and query strategies to support Human-in-the-Loop systems
  • Advanced Adaptive Learning approaches that use machine learning to optimize each step in the Human-in-the-Loop process
  • Real-world use cases from well-known data scientists

about the author

Robert (Munro) Monarch has built Annotation, Active Learning, and machine learning systems with machine learning-focused startups and with larger companies including Amazon, Google, IBM, and most major phone manufacturers. If you speak to your phone, if your car parks itself, if your music is tailored to your taste, or if your news articles are recommended for you, then there is a good chance that Robert contributed to this experience.

Robert holds a PhD from Stanford focused on Human-in-the-Loop machine learning for healthcare and disaster response, and is a disaster response professional in addition to being a machine learning professional. A worked example throughout this text is classifying disaster-related messages from real disasters that Robert has helped respond to in the past.

FREE domestic shipping on orders of three or more print books

For me this book is an eye opening for a new topic: how is possible to improve the results of my ML application in the context of the actual evolution of these technologies?

Alessandro Puzielli

I am very excited that this book exists. It's unlike any other book I've seen on the subject and addresses some very important topics.

Dana Robinson

This book covers an extremely important, yet woefully under-discussed topic. I feel like a lot of ML/DL projects would be more successful if they took the concepts presented here to heart.

Dana Robinson

A very clear demonstration of active learning applied to real-world problems.

Sayak Paul

This book is one of the first in the subject and shines light on an often ignored aspect of machine/deep learning which is the data annotation. This book should be a must read for any practitioner in the field!

Vidhya Vinay

I would strongly recommend this book for those of you who are interested to know about the different ways of data labelling, but don’t have the strong technical knowledge in this area.

Diego Poggioli