Human-in-the-loop machine learning centers on combining human judgment with machine learning to make AI systems more accurate, faster to improve, and more useful in real workflows. The chapter emphasizes that most modern AI still depends heavily on supervised learning and human feedback, whether the task is autonomous driving, translation, search relevance, or content moderation. It frames annotation, active learning, and transfer learning as the core techniques that let teams build effective systems without labeling everything from scratch.
A major focus of the chapter is annotation: turning raw data into training, validation, and evaluation data through human labeling. The text shows that annotation can be as simple as a web form or as complex as a rich graphical tool for tasks like drawing bounding boxes or polygons. Because label quality is so important, the chapter explains why annotation is hard: humans make mistakes, disagreements are common, subjective tasks are especially difficult, and poor labels can introduce serious bias that no algorithm can fully recover from. It also stresses that good annotation depends on both the right workforce and the right interface design, with attention to usability, keyboard efficiency, and avoiding priming effects.
The chapter also introduces active learning as a way to reduce labeling cost and speed up model improvement by selecting the most informative unlabeled items for annotation. It highlights uncertainty sampling, diversity sampling, and random sampling as the main strategies, and explains that they work best together in iterative cycles of labeling and retraining. Finally, it presents transfer learning as a way to reuse knowledge from pretrained models so that new tasks need fewer labels, especially in computer vision and increasingly in NLP. Throughout, the chapter argues that the best systems treat algorithms, annotation, and human-computer interaction as interconnected parts of one workflow rather than separate problems.
A mental model of the human-in-the-loop process for predicting labels on data
Pros and cons of different active learning strategies. Top left: The decision boundary from a machine learning algorithm between items, with some items labeled A and some labeled B. Top right: One possible result from uncertainty sampling. This active learning strategy is effective for selecting unlabeled items near the decision boundary. These items are the most likely to be wrongly predicted, and therefore, the most likely to get a label that moves the decision boundary. If all the uncertainty is in one part of the problem space, however, giving these items labels will not have a broad effect on the model. Bottom left: One possible result of diversity sampling. This active learning strategy is effective for selecting unlabeled items in different parts of the problem space. If the diversity is away from the decision boundary, however, these items are unlikely to be wrongly predicted, so they will not have a large effect on the model when a human gives them a label that is the same as the model predicted. Bottom right: One possible result from combining uncertainty sampling and diversity sampling. When the strategies are combined, items are selected that are near diverse sections of the decision boundary. Therefore, we are optimizing the chance of finding items that are likely to result in a changed decision boundary.
The iterative active learning process. Top left to bottom right: Two iterations of active learning. In each iteration, items are selected along a diverse selection of the boundary, which in turn causes the boundary to move after retraining, resulting in a more accurate machine learning model. Ideally, we requested human labels for the minimum number of items as part of our active learning strategy. This request speeds the time to get an accurate model and reduces the overall cost of human annotation.
An example of transfer learning. A model was built to predict a label as “A,” “B,” “C,” or “D.” Retraining the last layer of the model and using far fewer human-labeled items than if we were training a model from scratch, the model is able to predict labels “Y” and “Z.”
A machine learning knowledge quadrant, covering the topics in this book and expressing them in terms of what is known and unknown for your machine learning models
Summary
- The broader human-in-the-loop machine learning architecture is an iterative process combining human and machine components. Understanding these components explains how the parts of this book come together.
- You can use some basic annotation techniques to start creating training data. Understanding these techniques ensures that you are getting annotations accurately and efficiently.
- The two most common active learning strategies are uncertainty sampling and diversity sampling. Understanding the basic principles of each type helps you strategize about the right combination of approaches for your particular problems.
- Human–computer interaction gives you a framework for designing the user-experience components of human-in-the-loop machine learning systems.
- Transfer learning allows us to adapt models trained from one task to another and build more accurate models with fewer annotations.
Human-in-the-Loop Machine Learning ebook for free
