This chapter shifts from modeling on clean data to the realities of preparing messy, high-dimensional, real-world data for machine learning. It centers on vtreat, an R package that automates principled data preparation: handling missing values, outliers, categorical variables (including rare and novel levels), high-cardinality features, redundancy, and the risks of overfitting when variables are numerous. The authors motivate the workflow with a customer-churn task from the KDD Cup 2009 data, and stress sound data-splitting strategies and cross-validated preparation to avoid “nested model bias,” where information from the outcome leaks into feature construction and inflates apparent performance.
The vtreat workflow has two phases: design and application. In design, vtreat learns a treatment plan that captures safe, informative recodings of each variable; in application, it transforms any dataset into an all-numeric, no-missing-values frame. It produces derived variables such as cleaned numerics, “isBAD” indicators for missingness, one-hot level indicators for common categories, impact-coded summaries (single numeric encodings of a category’s relationship to the outcome), and prevalence measures. A “score frame” reports per-variable utility and complexity, guiding variable selection. For small or complex problems, vtreat’s cross-frame experiments generate cross-validated training features, aligning in-sample estimates with out-of-sample behavior. The chapter demonstrates building and evaluating a churn model (AUC-based assessment, thresholding for lift/recall trade-offs) and shows how the same ideas apply to regression.
To help readers “master” vtreat, the chapter walks through the core mechanics: treating missingness by adding indicators, explicit one-hot encoding, impact coding with smoothing to stabilize small-sample estimates, and disciplined prevention of outcome leakage via cross-frames or three-way splits. It discusses when to use designTreatments versus mkCrossFrame experiments, illustrates quick prototyping and data inspection habits, and flags ethical considerations around sensitive variables. An auto-mpg regression example reinforces that the approach generalizes beyond classification. The overarching takeaway is a robust, reproducible pipeline that converts messy data into modeling-ready features with realistic, cross-validated estimates of variable value, improving model reliability and downstream decision-making.
Figure 8.1. Mental model
Figure 8.2. vtreat 3-way split strategy
Figure 8.3. KDD2009 Churn Rate
Figure 8.4. vtreat variable preperation
Figure 8.5. vtreat variable preperation
Figure 8.6. vtreat 3 way split strategy again
Figure 8.7. vtreat cross-frame strategy
Figure 8.8. Distribution of the glm model's scores on test data
Figure 8.9. glm Recall and Enrichment as a Function of Threshold
Figure 8.10. The first few rows of the auto_mpg data
Figure 8.11. The two vtreat phases
Figure 8.12. Our simple example data: raw
Figure 8.13. Our simple example data: treated
Summary
Real-world data is often "messy." Raw un-inspected and un-treated data may crash your modeling or predicting step, or may give bad results. "Fixing" data of course does not compete with having better data. But being able to work with the data you have (instead of the data you want) is an advantage.
In addition to many domain-specific or problem-specific problems, you may find in your data, there are a number of common problems that should be anticipated and dealt with systematically. vtreat is a package specialized for preparing data for supervised machine learning or predictive modeling tasks. It can also reduce your project documentation requirements through its citable documentation. [70] However, remember that tools are not an excuse to avoid looking at your data.
In this chapter, you have learned:
- How to use the
vtreatpackage'sdesignTreatments*()/prepare()pattern with a three-way split of your training data to prepare messy data for model fitting and model application.
- How to use the
vtreatpackage'smkCrossFrame*Experiment()/prepare()pattern with a two-way split of your training data to prepare messy data for model fitting and model application, when statistical efficiency is important.
Practical Data Science with R, Second Edition ebook for free
