FREE
You can see this entire book for free.Click the table of contents to start reading.
The most comprehensive coverage of Dask to date, with real-world examples that made a difference in my daily work.
Dask is a native parallel analytics tool designed to integrate seamlessly with the libraries you’re already using, including Pandas, NumPy, and Scikit-Learn. With Dask you can crunch and work with huge datasets, using the tools you already have. And Data Science with Python and Dask is your guide to using Dask for your data projects without changing the way you work!
About the Technology
An efficient data pipeline means everything for the success of a data science project. Dask is a flexible library for parallel computing in Python that makes it easy to build intuitive workflows for ingesting and analyzing large, distributed datasets. Dask provides dynamic task scheduling and parallel collections that extend the functionality of NumPy, Pandas, and Scikit-learn, enabling users to scale their code from a single laptop to a cluster of hundreds of machines with ease.
About the book
Data Science with Python and Dask teaches you to build scalable projects that can handle massive datasets. After meeting the Dask framework, you’ll analyze data in the NYC Parking Ticket database and use DataFrames to streamline your process. Then, you’ll create machine learning models using Dask-ML, build interactive visualizations, and build clusters using AWS and Docker.
Table of Contents takes you straight to the bookdetailed table of contents
Part 1: The building blocks of scalable computing
1 Why scalable computing matters
1.1 Why Dask?
1.2 Cooking with DAGs
1.3 Scaling out, concurrency, and recovery
1.3.1 Scale Up vs. Scale Out
1.3.2 Concurrency and resource management
1.3.3 Recovering from failures
1.4 Introducing the companion dataset
Summary
2 Introducing Dask
2.1 Hello Dask: A first look at the DataFrame API
2.2 Visualizing DAGs
2.3 Lazy Computations
2.3.1 Data Locality
Summary
Part 2: Working with structured data using Dask DataFrames
3 Introducing Dask DataFrames
3.1 Why Use DataFrames?
3.2 Dask and Pandas
3.2.1 Managing DataFrame Partitioning
3.2.2 What is the Shuffle?
3.3 Limitations of Dask DataFrames
Summary
4 Loading data into DataFrames
4.1 Reading data from text files
4.1.1 Using Dask Datatypes
4.1.2 Creating Schemas for Dask DataFrames
4.2 Reading data from relational databases
4.3 Reading data from HDFS and S3
4.4 Reading data in Parquet format
Summary
5 Cleaning and transforming DataFrames
5.1 Working with indexes and axes
5.2 Dealing with missing values
5.3 Recoding data
5.4 Elementwise operations
5.5 Filtering and reindexing DataFrames
5.6 Joining and concatenating DataFrames
5.7 Writing data to text files and Parquet files
5.7.1 Writing to delimited text files
5.7.2 Writing to parquet files
Summary
6 Summarizing and analyzing DataFrames
6.1 Descriptive statistics
6.2 Built-in aggregate functions
6.3 Custom aggregate functions
6.4 Rolling (window) functions
Summary
7 Visualizing DataFrames with Seaborn
7.1 The prepare-reduce-collect-plot pattern
7.2 Visualizing continuous relationships with scatterplot and regplot
7.2.1 Creating a scatterplot with Dask and Seaborn
7.2.2 Adding a linear regression line to the scatterplot
7.2.3 Adding a nonlinear regression line to a scatterplot
7.3 Visualizing Categorical Relationships with violinplot
7.3.1 Creating a violinplot with Dask and Seaborn
7.3.2 Randomly sampling data from a Dask DataFrame
7.4 Visualizing two categorical relationships with heatmap
Summary
8 Visualizing Location Data with Datashader
8.1 What is Datashader and how does it work?
8.1.1 The five stages of the Datashader rendering pipeline
8.1.2 Creating a Datashader Visualization
8.2 Plotting location data as an interactive heatmap
8.2.1 Preparing geographic data for map tiling
8.2.2 Creating the interactive heatmap
Summary
Part 3: Extending and deploying Dask
9 Working with Bags and Arrays
9.1 Reading and parsing unstructured data with Bags
9.1.1 Selecting and viewing data from a Bag
9.1.2 Common parsing issues and how to overcome them
9.1.3 Working with delimiters
9.2 Transforming, filtering, and folding elements
9.2.1 Transforming elements with the map method
9.2.2 Filtering Bags with the filter method
9.2.3 Calculating descriptive statistics on Bags
9.2.4 Creating aggregate functions using the foldby method
9.3 Building Arrays and DataFrames from Bags
9.4 Using Bags for parallel text analysis with NLTK
9.4.1 The basics of bigram analysis
9.4.2 Extracting tokens and filtering stopwords
9.4.3 Analyzing the bigrams
Summary
10 Machine learning with Dask-ML
10.1 Building linear models with Dask-ML
10.1.1 Preparing the data with binary vectorization
10.1.2 Building a logistic regression model with Dask-ML
10.2 Evaluating and tuning Dask-ML models
10.2.1 Evaluating Dask-ML models with the score method
10.2.2 Building a naïve Bayes classifier with Dask-ML
10.2.3 Automatically tuning hyperparameters
10.3 Persisting Dask-ML models
Summary
11 Scaling and deploying Dask
11.1 Building a Dask cluster on Amazon AWS with Docker
11.1.1 Getting started
11.1.2 Creating a security key
11.1.3 Creating the ECS cluster
11.1.4 Configuring the cluster’s networking
11.1.5 Creating a shared data drive in Elastic File System
11.1.6 Allocating space for Docker images in Elastic Container Repository
11.1.7 Building and deploying images for scheduler, worker, and notebook
11.1.8 Connecting to the cluster
11.2 Running and monitoring Dask jobs on a cluster
11.3 Cleaning up the Dask cluster on AWS
Summary
Appendixes
Appendix A: Software Installation
A.1 Installing additional packages with Anaconda
A.2 Installing packages without Anaconda
A.3 Starting a Jupyter Notebook server
A.4 Configuring NLTK
What's inside
- Working with large, structured and unstructured datasets
- Visualization with Seaborn and Datashader
- Implementing your own algorithms
- Building distributed apps with Dask Distributed
- Packaging and deploying Dask apps
About the reader
For data scientists and developers with experience using Python and the PyData stack.
About the author
Jesse Daniel is an experienced Python developer. He taught Python for Data Science at the University of Denver and leads a team of data scientists at a Denver-based media technology company.
We interviewed Jesse as a part of our Six Questions series. Check it out here.
