Welcome to Manning India!
We are pleased to be able to offer regional eBook pricing for Indian residents.
All eBook prices are discounted 40% or more!
Deep Learning with PyTorch
FREE
You can see any available part of this book for free.Click the table of contents to start reading.
Every other day we hear about new ways to put deep learning to good use: improved medical imaging, accurate credit card fraud detection, long range weather forecasting, and more. PyTorch puts these superpowers in your hands, providing a comfortable Python experience that gets you started quickly and then grows with you as you—and your deep learning skills—become more sophisticated. Deep Learning with PyTorch will make that journey engaging and fun.Perfectly motivating and practical without being shallow.
Table of Contents takes you straight to the bookdetailed table of contents
Part 1: Core PyTorch
1 Introducing Deep Learning and the PyTorch Library
1.1 What is PyTorch?
1.2 What is this book?
1.3 Why PyTorch
1.3.1 The Deep Learning Revolution
1.3.2 Immediate vs. deferred execution
1.3.3 The deep learning competitive landscape
1.4 PyTorch has the batteries included
1.4.1 Hardware for deep learning
1.4.2 Using Jupyter notebooks
1.5 Conclusion
1.6 Exercises
1.7 Summary
2 Pre-Trained Networks
2.1 A pre-trained network that recognizes the subject of an image
2.1.1 Obtaining a pre-trained network for image recognition
2.1.2 AlexNet
2.1.3 ResNet
2.1.4 Ready, set, almost run
2.1.5 Run!
2.2 A pre-trained model that fakes it until it makes it
2.2.1 The GAN game
2.2.2 CycleGAN
2.2.3 A network that turns horses into zebras
2.3 A pre-trained network that describes scenes
2.3.1 NeuralTalk2
2.4 Torch Hub
2.5 Conclusion
2.6 Exercises
2.7 Summary
3 It Starts with a Tensor
3.1 Tensors are multi-dimensional arrays
3.1.1 From Python lists to PyTorch tensors
3.1.2 Constructing our first tensors
3.1.3 The essence of tensors
3.2 Indexing Tensors
3.3 Named Tensors
3.4 Tensor element types
3.4.1 Specifying the numeric type with dtype
3.4.2 A dtype for every occasion
3.4.3 Managing a tensor’s dtype attribute
3.5 The tensor API
3.6 Tensors — scenic views on storage
3.6.1 Indexing into storage
3.6.2 Modifying Stored Values — Inplace Operations
3.7 Tensor metadata: size, offset, stride
3.7.1 Views over another tensor’s storage
3.7.2 Transposing without copying
3.7.3 Transposing in higher dimensions
3.7.4 Contiguous tensors
3.8 Moving tensors to the GPU
3.8.1 Managing a tensor’s device attribute
3.9 NumPy interoperability
3.10 Generalized Tensors are Tensors, too
3.11 Serializing tensors
3.11.1 Serializing to HDF5 with h5py
3.12 Conclusion
3.13 Exercises
3.14 Summary
4 Real-World Data Representation Using Tensors
4.1 Images
4.1.1 Adding color channels
4.1.2 Loading an image file
4.1.3 Changing the layout
4.1.4 Normalizing the data
4.2 Volumetric Data
4.2.1 Loading a specialized format
4.3 Tabular Data
4.3.1 Using a real-world dataset
4.3.2 Loading a wine data tensor
4.3.3 Representing scores
4.3.4 One-hot encoding
4.3.5 When to categorize
4.3.6 Finding thresholds
4.4 Time Series
4.4.1 Adding a time dimension
4.4.2 Shaping the data by time period
4.4.3 Ready for training
4.5 Text
4.5.1 Converting text to numbers
4.5.2 One-hot encoding characters
4.5.3 One-hot encoding whole words
4.5.4 Text embeddings
4.5.5 Text embeddings as a blueprint
4.6 Conclusion
4.7 Exercises
4.8 Summary
5 The Mechanics of Learning
5.1 Learning is just parameter estimation
5.1.1 A Hot Problem
5.1.2 Gathering some data
5.1.3 Visualizing the data
5.1.4 Choosing a linear model as a first try
5.2 Less loss is what we want
5.2.1 From Problem back to PyTorch
5.2.2 Broadcasting
5.3 Down Along the Gradient
5.3.1 Decreasing loss
5.3.2 Getting Analytical
5.3.2.1 Computing the derivatives
5.3.2.2 Applying the derivatives to the model
5.3.2.3 Defining the gradient function
5.3.3 Iterating to fit the model
5.3.3.1 The Training Loop
5.3.3.2 Overtraining
5.3.4 Normalizing inputs
5.3.5 Visualizing (again)
5.4 PyTorch’s Autograd: Back-propagate all things
5.4.1 Computing the gradient automatically
5.4.1.1 Applying autograd
5.4.1.2 Using the grad attribute
5.4.1.3 Accumulating grad functions
5.4.2 Optimizers a-la Carte
5.4.2.1 Using a gradient descent optimizer
5.4.2.2 Testing other optimizers
5.4.3 Training, Validation, and Overfitting
5.4.3.1 Evaluating the training loss
5.4.3.2 Generalizing to the validation set
5.4.3.3 Splitting a dataset
5.4.4 Autograd Nits and Switching it Off
5.5 Conclusion
5.6 Exercises
5.7 Summary
6 Using A Neural Network To Fit the Data
6.1 Artificial Neurons
6.1.1.1 Composing a multi-layer network
6.1.1.2 Understanding the error function
6.1.2 All We Need is Activation
6.1.2.1 Capping the output range
6.1.2.2 Compressing the output range
6.1.2.3 More activation functions
6.1.2.4 Choosing the best activation function
6.1.3 What learning means for a neural network
6.2 The PyTorch nn module
6.2.1.1 Using call rather than forward
6.2.1.2 Returning to the linear model
6.2.1.3 Batching inputs
6.2.1.4 Optimizing batches
6.2.2 Finally a Neural Network
6.2.2.1 Replacing the linear model
6.2.2.2 Inspecting the parameters
6.2.2.3 Comparing to the linear model
6.3 Conclusion
6.4 Exercises
6.5 Summary
7 Telling Birds from Airplanes: Learning from Images
7.1 A dataset of tiny images
7.1.1 Downloading CIFAR10
7.1.2 The Dataset class
7.1.3 Dataset transforms
7.1.4 Normalizing data
7.2 Distinguishing birds from airplanes
7.2.1 Building the dataset
7.2.2 A fully connected model
7.2.3 Output of a classifier
7.2.4 Representing the output as probabilities
7.2.5 A loss for classifying
7.2.6 Training the classifier
7.2.7 The limits of going fully connected
7.3 Conclusion
7.4 Exercises
7.5 Summary
8 Using Convolutions To Generalize
8.1 The case for convolutions
8.1.1 What convolutions do exactly
8.2 Convolutions in action
8.2.1 Padding the boundary
8.2.2 Detecting features with convolutions
8.2.3 Looking further with depth and pooling
8.2.3.1 From large to small: Downsampling
8.2.3.2 Combining convolutions and downsampling for great good
8.2.4 Putting it all together for our network
8.3 Subclassing nn.Module
8.3.1 Our network as a nn.Module
8.3.2 How PyTorch keeps track of parameters and submodules
8.3.3 The Functional API
8.4 Training our Convnet
8.4.1 Measuring accuracy
8.4.2 Saving and loading our model
8.4.3 Training on the GPU
8.4.4 Summary
8.5 Model Design
8.5.1 Adding memory capacity: Width
8.5.2 Helping our model to converge and generalize: Regularization
8.5.2.1 Keeping the parameters in check: Weight penalties
8.5.2.2 Not relying too much on a single input: Dropout
8.5.2.3 Keeping activations in check: Batch normalization
8.5.3 Going deeper to learn more complex structures: Depth
8.5.3.1 Skip connections
8.5.3.2 Building very deep models in PyTorch
8.5.3.3 Initialization
8.5.4 Comparing the designs of this section
8.5.5 Now it’s already outdated
8.6 Conclusion
8.7 Exercises
8.8 Summary
Part 2: Learning from Images in the Real-World: Early Detection of Lung Cancer
9 Using PyTorch To Fight Cancer
9.1 What is a CT scan, exactly?
9.2 The project: an end-to-end malignancy detector for lung cancer
9.2.1 Why can’t we just throw data at a neural network until it works?
9.2.2 What is a nodule?
9.2.3 Our data source: the LUNA Grand Challenge
9.2.4 How to download the LUNA data
9.3 Conclusion
9.4 Summary
10 Ready, Dataset, Go!
10.1 Parsing LUNA’s annotation data
10.2 Loading individual CT scans
10.2.1 Hounsfield Units
10.3 Locating a nodule using the patient coordinate system
10.3.1 Extracting a nodule from a CT scan
10.4 A straightforward Dataset implementation
10.4.1 Caching nodule arrays with the getCtRawNodule function
10.4.2 Constructing our dataset in LunaDataset.{uu}init{uu}
10.4.3 A Training / Validation Split
10.4.4 Rendering the data
10.5 Conclusion
10.6 Exercises
10.7 Summary
11 Training A Classification Model To Detect Suspected Tumors
11.1 The main entrypoint for our application
11.2 Pre-training setup and initialization
11.2.1 Initalizing the model and optimizer
11.2.2 Care and feeding of DataLoaders
11.3 Our first-pass neural network design
11.3.1 The Core Convolutions
11.3.2 The Full Model
11.3.2.1 Complication: Converting from convolution to linear
11.3.2.2 Initialization
11.4 Training and validating the model
11.4.1 The computeBatchLoss function
11.4.2 The validation loop is similar
11.5 Outputting performance metrics
11.5.1 The logMetrics function
11.5.1.1 Constructing Masks
11.6 Running the training script
11.6.1 Needed data for training
11.6.2 Interlude: the enumerateWithEstimate function
11.7 Evaluating the model: Getting 99.7% correct means we’re done, right?
11.8 Graphing training metrics with TensorBoard
11.8.1 Running TensorBoard
11.8.2 Adding tensorboard support to our metrics logging function
11.8.2.1 Writing scalars to TensorBoard
11.9 Visdom, a TensorBoard alternative
11.10 Why is the model not learning to detect nodules?
11.11 Conclusion
11.12 Exercises
11.13 Summary
12 Monitoring Metrics: Precision, Recall, and Pretty Pictures
12.1 Good dogs versus bad guys: false positives and false negatives
12.2 Graphing the positives and negatives
12.2.1 Recall is Roxie’s strength
12.2.2 Precision is Peston’s forte
12.2.3 Implementing precision and recall in logMetrics
12.2.4 Our ultimate performance metric: the F1 score
12.2.4.1 Updating the logging output to include precision, recall, and F1 score
12.2.5 How does our model perform with our new metrics?
12.3 What does an ideal data set look like
12.3.1 Making the data look less like the actual and more like the "ideal"
12.3.1.1 Samplers Can Reshape Data Sets
12.3.2 Implementing class balancing
12.3.3 Contrasting training with a balanced LunaDataset to previous runs
12.3.4 Recognizing the symptoms of overfitting
12.4 Revisiting the problem of over-fitting
12.4.1 An over-fit face-to-age prediction model
12.5 Preventing Overfitting with Data Augmentation
12.5.1 Specific Data Augmentation Techniques
12.5.1.1 Mirroring
12.5.1.2 Shifting by a random offset
12.5.1.3 Scaling
12.5.1.4 Rotating
12.5.1.5 Noise
12.5.1.6 Examining augmented candidates
12.5.2 Seeing the improvement from data augmentation
12.6 Conclusion
12.7 Exercises
12.8 Summary
13 Using Segmentation To Find Suspected Nodules
13.1 Segmentation is per-pixel classification
13.1.1 The UNet architecture
13.2 Updating the model for segmentation
13.2.1 Adapting an off-the-shelf model to our project
13.3 Updating the Dataset for segmentation
13.3.1 UNet has very specific input size requirements
13.3.2 UNet tradeoffs for 3D vs. 2D data
13.3.2.1 Handling tradeoffs
13.3.3 Building the ground truth data
13.3.3.1 Bounding boxes
13.3.3.2 Calling mask creation during Ct init
13.3.3.3 Caching chunks of the mask in addition to the CT
13.3.3.4 Implementing the lung mask
13.3.4 Implementing the Luna2dSegmentationDataset
13.3.5 Implementing the TrainingLuna2dSegmentationDataset
13.3.6 Augmenting on the GPU
13.4 Updating the training script for segmentation
13.4.1 Initializing our segmentation and augmentation models
13.4.2 Using the Adam optimizer
13.4.3 Dice loss
13.4.3.1 Loss weighting
13.4.3.2 Metrics collection
13.4.4 Getting images into TensorBoard
13.4.5 Updating our metrics logging
13.4.6 Saving our model
13.5 Results
13.6 Conclusion
13.7 Exercises
13.8 Summary
14 Clustering and Diagnosis
14.1 From CT to Nodules: Stiching segmentation and classification together
14.1.1 Independence of the validation set
14.1.2 Implementing the CT processing
14.1.3 Segmentation
14.1.4 Clustering
14.1.5 Did we find a nodule or not so much: Classification to reduce false positives
14.2 Quantitative validation
14.3 Predicting Malignancy
14.3.1 Getting malignancy information
14.3.2 A baseline: Classifying by Diameter
14.3.3 Re-using pre-existing weights: Finetuning
14.3.4 Varying our model design
14.3.4.1 Writing histograms to TensorBoard
14.3.5 Results
14.4 What do we see when we diagnose
14.5 Additional sources of inspiration (and data)
14.5.1 The Data Science Bowl 2017
14.5.2 LUNA papers
14.6 Conclusion
14.6.1 Behind the curtain
14.7 Exercises
14.8 Summary
Part 3: Deploying PyTorch Models
15 Deploying to production
15.1 Serving PyTorch models
15.1.1 Our model behind a Flask server
15.1.2 What we wish from Deployment
15.1.3 Request Batching
15.2 Exporting Models
15.2.1 Interoperability beyond PyTorch with ONNX
15.2.2 PyTorch’s own export — tracing
15.2.3 Our server with a traced model
15.3 Interacting with the PyTorch JIT
15.3.1 What to expect from moving beyond classic Python/PyTorch
15.3.2 The dual nature of PyTorch as Interface and Backend
15.3.3 TorchScript
15.3.4 Scripting the gaps of traceability
15.3.5 Using the JIT to pass things to other backends
15.4 LibTorch — PyTorch in C++
15.4.1 Running JITed models from C++
15.4.2 C from the start — the C API
15.5 Emerging Technology: Enterprise serving of PyTorch models
15.6 Going mobile
15.6.1 Improving efficiency: Model design and quantization
15.7 Conclusion
15.8 Exercises
15.9 Summary
About the Technology
PyTorch is a machine learning framework with a strong focus on deep neural networks. Because it emphasizes GPU-based acceleration, PyTorch performs exceptionally well on readily-available hardware and scales easily to larger systems. Plus it’s Pythonic! Thanks to its define-by-run computation graph model, PyTorch plays nicely with the Python data science ecosystem. It’s instantly familiar if you’re using Numpy, Pandas, or other similar tools.It’s easy to get started with PyTorch. It minimizes cognitive overhead without sacrificing the access to advanced features, meaning you can focus on what matters the most - building and training the latest and greatest deep learning models and contribute to making a dent in the world. PyTorch is also a snap to scale and extend, and it partners well with other Python tooling. PyTorch has been adopted by hundreds of deep learning practitioners and several first-class players like FAIR, OpenAI, FastAI and Purdue.
About the book
Deep Learning with PyTorch teaches you how to implement deep learning algorithms with Python and PyTorch. This book takes you into a fascinating case study: building an algorithm capable of detecting malignant lung tumors using CT scans. As the authors guide you through this real example, you'll discover just how effective and fun PyTorch can be. After a quick introduction to the deep learning landscape, you'll explore the use of pre-trained networks and start sharpening your skills on working with tensors. You'll find out how to represent the most common types of data with tensors and how to build and train neural networks from scratch on practical examples, focusing on images and sequences.After covering the basics, the book will take you on a journey through larger projects. The centerpiece of the book is a neural network designed for cancer detection. You'll discover ways for training networks with limited inputs and start processing data to get some results. You'll sift through the unreliable initial results and focus on how to diagnose and fix the problems in your neural network. Finally, you'll look at ways to improve your results by training with augmented data, make improvements to the model architecture, and perform other fine tuning.
What's inside
- Using the PyTorch tensor API
- Understanding automatic differentiation in PyTorch
- Training deep neural networks
- Monitoring training and visualizing results
- Implementing modules and loss functions
- Loading data in Python for PyTorch
- Interoperability with NumPy
- Deploying a PyTorch model for inference
About the reader
Written for developers with some knowledge of Python as well as basic linear algebra skills. Some understanding of deep learning will be helpful, however no experience with PyTorch or other deep learning frameworks is required.About the authors
Eli Stevens has worked in Silicon Valley for the past 15 years as a software engineer, and the past 7 years as Chief Technical Officer of a startup making medical device software. Luca Antiga is co-founder and CEO of an AI engineering company located in Bergamo, Italy, and a regular contributor to PyTorch. Thomas Viehmann is a Machine Learning and PyTorch speciality trainer and consultant based in Munich, Germany and a PyTorch core developer.
Deep Learning with PyTorch authors Luca Antiga (L) and Eli Stevenson (R) eating dessert in San Francisco's Mission District with the book's editor Frances Lefkowitz. Luca is from Bergamo, Italy, Eli lives in San Jose, and Frances hails from San Francisco.
Manning Early Access Program (MEAP)
Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
Prices displayed in rupees will be charged in USD when you check out.
placing your order...
Don't refresh or navigate away from the page.FREE domestic shipping on three or more pBooks