Deep learning is applicable to a widening range of artificial intelligence problems, such as image classification, speech recognition, text classification, question answering, text-to-speech, and optical character recognition. It is the technology behind photo tagging systems at Facebook and Google, self-driving cars, speech recognition systems on your smartphone, and much more.
In particular, Deep learning excels at solving machine perception problems: understanding the content of image data, video data, or sound data. Here's a simple example: say you have a large collection of images, and that you want tags associated with each image, for example, "dog," "cat," etc. Deep learning can allow you to create a system that understands how to map such tags to images, learning only from examples. This system can then be applied to new images, automating the task of photo tagging. A deep learning model only has to be fed examples of a task to start generating useful results on new data.
"Examples chosen are not only useful but also highlight the flexibility of deep learning in solving problems in many different fields. The author makes a great job of both introducing the reader to deep learning and the Keras library. "
~ Gustavo Patino
"All of the major topics and concepts of DL are covered and well explained, using code examples and diagrams, instead of mathematical formulas."
~ Srdjan Santic
Part 1: The fundamentals of Deep Learning
1. What is Deep Learning?
1.1. Artificial intelligence, machine learning and deep learning
1.1.1. Artificial intelligence
1.1.2. Machine Learning
1.1.3. Learning representations from data
1.1.4. The "deep" in deep learning
1.1.5. Understanding how deep learning works in three figures
1.1.6. What deep learning has achieved so far
1.1.7. Don’t believe the short-term hype
1.1.8. The promise of AI
1.2. Why deep learning, why now?
1.2.1. Hardware
1.2.2. Data
1.2.3. Algorithms
1.2.4. A new wave of investment
1.2.5. The democratization of deep learning
1.2.6. Will it last?
2. Before we start: the mathematical blocks of neural networks
2.1. A first look at a neural network
2.2. Data representations for neural networks
2.2.1. Scalars (0D tensors)
2.2.2. Vectors (1D tensors)
2.2.3. Matrices (2D tensors)
2.2.4. 3D tensors and higher-dimensional tensors
2.2.5. Key attributes
2.2.6. Manipulating tensors in Numpy
2.2.7. The notion of data batch
2.2.8. Real-world examples of data tensors
2.2.9. Vector data
2.2.10. Timeseries data or sequence data
2.2.11. Image data
2.2.12. Video data
2.3. The gears of neural networks: tensor operations
2.3.1. Element-wise operations
2.3.2. Broadcasting
2.3.3. Tensor dot
2.3.4. Tensor reshaping
2.3.5. Geometric interpretation of tensor operations
2.3.6. A geometric interpretation of deep learning
2.4. The engine of neural networks: gradient-based optimization
2.4.1. What's a derivative?
2.4.2. Derivative of a tensor operation: the gradient
2.4.3. Stochastic gradient descent
2.4.4. Chaining derivatives: the backpropagation algorithm
2.4.5. In summary: training neural networks using gradient descent
2.5. Looking back on our first example
3. Getting started with neural networks
3.1. Anatomy of a neural network
3.1.1. Layers: the Lego bricks of deep learning
3.1.2. Models: networks of layers
3.1.3. Loss functions and optimizers: keys to configuring the learning process
3.2. Introduction to Keras
3.2.1. Keras, TensorFlow, Theano, and CNTK
3.2.2. Developing with Keras: a quick overview
3.3. Setting up a deep learning workstation
3.3.1. Preliminary considerations
3.3.2. Jupyter notebooks: the prefered way to run deep learning experiments
3.3.3. Getting Keras running: two options
3.3.4. Running deep learning jobs in the cloud: pros and cons
3.3.5. What is the best GPU for deep learning?
3.4. Classifying movie reviews: a binary classification example
3.4.1. The IMDB dataset
3.4.2. Preparing the data
3.4.3. Building our network
3.4.4. Validating our approach
3.4.5. Using a trained network to generate predictions on new data
3.4.6. Further experiments
3.4.7. Conclusions
3.5. Classifying newswires: a multi-class classification example
3.5.1. The Reuters dataset
3.5.2. Preparing the data
3.5.3. Building our network
3.5.4. Validating our approach
3.5.5. Generating predictions on new data
3.5.6. A different way to handle the labels and the loss
3.5.7. On the importance of having sufficiently large intermediate layers
3.5.8. Further experiments
3.5.9. Conclusions
3.6. Predicting house prices: a regression example
3.6.1. The Boston Housing Price dataset
3.6.2. Preparing the data
3.6.3. Building our network
3.6.4. Validating our approach using K-fold validation
3.6.5. Conclusions
4. Fundamentals of machine learning
4.1. Four different brands of machine learning
4.1.1. Supervised learning.
4.1.2. Unsupervised learning.
4.1.3. Self-supervised learning.
4.1.4. Reinforcement learning.
4.2. Classification and regression glossary
4.3. Evaluating machine learning models
4.3.1. Training, validation, and test sets
4.3.2. Simple hold-out validation
4.3.3. K-fold validation
4.3.4. Iterated K-fold validation with shuffling
4.3.5. Keep in mind…
4.4. Data preprocessing, feature engineering and feature learning
4.4.1. Data preprocessing for neural networks
4.4.2. Feature engineering
4.5. Overfitting and underfitting
4.5.1. Fighting overfitting
4.6. The universal workflow of machine learning
4.6.1. Define the problem and assemble a dataset
4.6.2. Pick a measure of success
4.6.3. Decide on an evaluation protocol
4.6.4. Prepare your data
4.6.5. Develop a model that does better than a baseline
4.6.6. Scale up: develop a model that overfits
4.6.7. Regularize your model and tune your hyperparameters
Part 2: Deep learning in practice
5. Deep learning for computer vision
5.1. Introduction to convnets
5.1.1. The convolution operation
5.1.2. The max pooling operation
5.2. Training a convnet from scratch on a small dataset
5.2.1. The relevance of deep learning for small-data problems
5.2.2. Downloading the data
5.2.3. Building our network
5.2.4. Data preprocessing
5.2.5. Using data augmentation
5.3. Using a pre-trained convnet
5.3.1. Feature extraction
5.3.2. Fine-tuning
5.3.3. Take-aways: using convnets with small datasets
5.4. Visualizing what convnets learn
5.4.1. Visualizing intermediate activations
5.4.2. Visualizing convnet filters
5.4.3. Visualizing heatmaps of class activation
5.5. Wrapping up: deep learning for computer vision
6. Deep learning for text and sequences
6.1. Working with text data
6.1.1. One-hot encoding of words or characters
6.1.2. Using word embeddings
6.1.3. Putting it all together: from raw text to word embeddings
6.1.4. Wrapping up
6.2. Understanding recurrent neural networks
6.2.1. A first recurrent layer in Keras
6.2.2. Understanding the LSTM and GRU layers
6.2.3. A concrete LSTM example in Keras
6.2.4. Wrapping up
6.3. Advanced usage of recurrent neural networks
6.3.1. A temperature forecasting problem
6.3.2. Preparing the data
6.3.3. A common sense, non-machine learning baseline
6.3.4. A basic machine learning approach
6.3.5. A first recurrent baseline
6.3.6. Using recurrent dropout to fight overfitting
6.3.7. Stacking recurrent layers
6.3.8. Using bidirectional RNNs
6.3.9. Going even further
6.3.10. Wrapping up
6.3.11. 1D Convnets as an alternative to RNNs for sequence processing
6.3.12. Understanding 1D convolution for sequence data
6.3.13. 1D Pooling for sequence data
6.3.14. Implementing a 1D convnet
6.3.15. Combining CNNs and RNNs to process long sequences
6.3.16. Wrapping up
6.4. Wrapping up: deep learning for text and sequences
7. Advanced deep learning best practices
7.1. Going beyond the Sequential model: the Keras functional API
7.1.1. Introduction to the functional API
7.1.2. Multi-input models
7.1.3. Multi-output models
7.1.4. Directed acyclic graphs of layers
7.1.5. Layer weight sharing
7.1.6. Models as layers
7.1.7. Wrapping up
7.2. Inspecting and monitoring deep learning models: using Keras callbacks and TensorBoard
7.2.1. Using callbacks to act on a model during training
7.2.2. Introduction to TensorBoard: the TensorFlow visualization framework
7.2.3. Wrapping up
7.3. Getting the most out of your models
7.3.1. Advanced architecture patterns
7.3.2. Hyperparameter optimization
7.3.3. Model ensembling
7.3.4. Wrapping up
7.4. Wrapping up: advanced deep learning best practices
8. Generative deep learning
8.1. Text generation with LSTM
8.1.1. A brief history of generative recurrent networks
8.1.2. How can we generate sequence data?
8.1.3. The importance of the sampling strategy
8.1.4. Implementing character-level LSTM text generation
8.2. Deep Dream
8.2.1. Implementing Deep Dream in Keras
8.2.2. Take aways
8.3. Neural style transfer
8.3.1. The content loss
8.3.2. The style loss
8.3.3. In short
8.3.4. Neural style transfer in Keras
8.3.5. Take aways
8.4. Generating images with Variational Autoencoders
8.4.1. Sampling from latent spaces of images
8.4.2. Concept vectors for image editing
8.4.3. Variational autoencoders
8.4.4. Take aways
8.5. Introduction to generative adversarial networks
8.5.1. A schematic GAN implementation
8.5.2. A bag of tricks
8.5.3. The generator
8.5.4. The discriminator
8.5.5. The adversarial network
8.5.6. How to train your DCGAN
8.5.7. Take-aways
8.6. Wrapping up: generative deep learning
9. Conclusions
9.1. Key concepts in review
9.1.1. Different brands of approaches to AI
9.1.2. What makes deep learning special within machine learning
9.1.3. How to think about deep learning
9.1.4. Key enabling technologies
9.1.5. The universal machine learning workflow
9.1.6. Key network architectures
9.1.7. The space of possibilities
9.1.8. Mapping image data to vector data
9.1.9. Mapping timeseries data to vector data
9.2. The limitations of deep learning
9.2.1. The risk of anthropomorphizing machine learning models
9.2.2. Local generalization versus extreme generalization
9.2.3. Take-aways
9.3. The future of deep learning
9.3.1. Models as programs
9.3.2. Beyond backpropagation and differentiable layers
9.3.3. Automated machine learning
9.3.4. Lifelong learning and modular subroutine reuse
9.3.5. In summary: the long-term vision
9.4. Staying up to date in a fast-moving field
9.4.1. Practice on real-world problems using Kaggle
9.4.2. Read about the latest developments on Arxiv
9.4.3. Explore the Keras ecosystem
9.5. Final words
About the Technology
Keras is one of the most widely used deep learning frameworks for Python. Keras puts ease of use and accessibility front and center, making it a great fit for getting started with deep learning. Besides being easy to work with, it is suitable for the most advanced use cases as well: almost all deep learning competitions on Kaggle.com are won using Keras code.
About the book
Deep Learning with Python is for developers with some Python experience who want to learn how to use deep learning to solve real-world problems. The book is structured around a series of practical code examples that illustrate each new concept introduced and demonstrate best practices. You begin by finding out what deep learning is, and how it connects with Artificial Intelligence and Machine Learning, as well as why deep learning is rapidly gaining in importance right now. You will then learn the fundamentals of machine learning, and finally, you will dive deeper into practical applications in computer vision, natural language processing, and more. By the time you reach the end of this book, you will have become a Keras expert and will be able to apply deep learning in your own projects.
What's inside
- Understanding key machine learning concepts
- Setting up a computer environment for deep learning
- Using convolutional neural networks to solve image classification tasks
- Understanding and visualizing the representations that neural networks learn
- Using recurrent neural networks to solve text and sequence classification tasks
- Using deep learning for image style transfer, text generation and image generation
- Written by the creator of Keras, the Python deep learning library
About the reader
Readers should have Python experience. No previous experience with machine learning or deep learning is required.About the author
Francois Chollet is the author of Keras, one of the most widely used libraries for deep learning in Python. He has been working with deep neural networks since 2012. Francois is currently doing deep learning research at Google. He blogs about deep learning at blog.keras.io.
FREE domestic shipping on three or more pBooks