Grokking Deep Learning
Andrew W. Trask
  • MEAP began August 2016
  • Publication in Summer 2017 (estimated)
  • ISBN 9781617293702
  • 325 pages (estimated)
  • printed in black & white

Artificial Intelligence is one of the most exciting technologies of the century, and Deep Learning is in many ways the "brain" behind some of the world's smartest Artificial Intelligence systems out there. Loosely based on neuron behavior inside of human brains, these systems are rapidly catching up with the intelligence of their human creators, defeating the world champion Go player, achieving superhuman performance on video games, driving cars, translating languages, and sometimes even helping law enforcement fight crime. Deep Learning is a revolution that is changing every industry across the globe.

Grokking Deep Learning is the perfect place to begin your deep learning journey. Rather than just learn the "black box" API of some library or framework, you will actually understand how to build these algorithms completely from scratch. You will understand how Deep Learning is able to learn at levels greater than humans. You will be able to understand the "brain" behind state-of-the-art Artificial Intelligence. Furthermore, unlike other courses that assume advanced knowledge of Calculus and leverage complex mathematical notation, if you're a Python hacker who passed high-school algebra, you're ready to go. And at the end, you'll even build an A.I. that will learn to defeat you in a classic Atari game.

Table of Contents detailed table of contents

Part 1: Neural Network Basics

1. Introducing Deep Learning

1.1. Welcome to Grokking Deep Learning

1.2. Why should you learn Deep Learning?

1.3. Why you should read this book!

1.4. What you need to get started

1.5. You’ll probably need some Python knowledge

1.6. Conclusion and Primer for Chapter 2

2. Fundamental Concepts

2.1. What is Deep Learning?

2.2. What is Machine Learning?

2.3. Supervised Machine Learning

2.4. Unsupervised Machine Learning

2.5. Parametric vs Non-Parametric Learning

2.6. Supervised Parametric Learning

2.7. Step 1: Predict

2.8. Step 2: Compare to Truth Pattern

2.9. Step 3: Learn the Pattern

2.10. Unsupervised Parametric Learning

2.11. Conclusion

3. Building Your First Neural Network

3.1. What am I going to learn in this chapter?

3.2. What is a Neural Network?

3.3. What does a Neural Network do?

3.4. Does the network make accurate predictions?

3.5. Why measure error?

3.6. What the Simplest Form of Neural Network Learning?

3.7. Characteristics of Hot and Cold Learning

3.8. Calculating Both direction and amount from error

3.9. Learning Is Just Reducing Error

3.10. Let’s Back Up And Talk about Functions

3.11. Tunnel Vision on One Concept

3.12. Relationship Exploration: Hot and Cold

3.13. A Box With Rods Poking Out of It

3.14. Derivatives…​ take Two

3.15. What you really need to know…​

3.16. What you don’t really need to know…​

3.17. How to use a derivative to learn

3.18. Where is our derivative in the code?

3.19. Learning Method: Gradient Descent

3.20. Breaking Gradient Descent

3.21. Divergence

3.22. Introducing…​. Alpha

3.23. Alpha In Code

3.24. Memorizing

4. Neural Networks with Multiple Inputs

4.1. Expanding to Multiple Input Values

4.2. What We’re Going to Build

4.3. The Street Light Problem

4.4. Preparing our Data

4.5. Matrices and the Matrix Relationship

4.6. Creating a Matrix or Two in Python

4.7. What is a Neural Network

4.8. How Does A Neural Network Predict?

4.9. How Does A Neural Network Learn?

4.10. Slow down! What is "delta"?

4.11. Building our Neural Network in Python

4.12. Teaching Our Neural Network in Python

4.13. Quick Review from 10,000 Feet

4.14. Putting it all together in Python (verbose version)

4.15. Putting it all together in Python (polished version)

4.16. The Differences Line by Line

4.17. Learning the whole dataset!

4.18. Write it from Memory!

5. Building Your First Deep Neural Network

5.1. Neural Networks Learn Correlation

5.2. Up and Down Pressure

5.3. Up and Down Pressure (cont.)

5.4. Edge Case: Overfitting

5.5. Edge Case: Conflicting Pressure

5.6. Edge Case: Conflicting Pressure (cont.)

5.7. Learning Indirect Correlation

5.8. Creating Our Own Correlation

5.9. Stacking Neural Networks?

5.10. Backpropagation: Long Distance Error Attribution

5.11. Backpropagation: Why does this work?

5.12. Linear vs Non-Linear

5.13. Why The Neural Network Still Doesn’t Work

5.14. The Secret to "Sometimes Correlation"

5.15. A Quick Break

5.16. Our New Prediction Code

5.17. What We Will Build

5.18. What It Looks Like

5.19. What It Looks Like

5.20. A Non-Linear Problem

5.21. A Non-Linear Solution

Part 2: Image Deep Learning

6. Human Brain Inspired Improvements to Neural Networks

7. Neural Networks that Understand Edges and Corners

8. Crazy Deep Networks for Image Recognition (and why you love them)

Part 3: Language Deep Learning (Text and Audio)

9. Neural Network Word Math (King - Man + Woman = Queen)

10. Translating English to Spanish + Writing Like Shakespeare

11. Build Your Own Dictation Neural Network

Part 4: Neural Networks that Play Games

12. Building a Neural Network that Destroys You in Pong

13. Superhuman Level Atari Playing Neural Networks


Appendix A: Frameworks (Torch, TensorFlow, Keras)

About the Technology

Artificial Intelligence is one of the most exciting technologies of the century, and Deep Learning is in many ways the "brain" behind some of the world?s smartest Artificial Intelligence systems out there.

What's inside

  • How neural networks "learn"
  • You will build neural networks that can see and understand images
  • You will build neural networks that can translate text between languages and even write like Shakespeare
  • You will build neural networks that can learn how to play videogames

About the reader

Written for readers with high school-level math and intermediate programming skills. Experience with Calculus is helpful but NOT required.

About the author

Andrew Trask is a PhD student at Oxford University, funded by the Oxford-DeepMind Graduate Scholarship, where he researches Deep Learning approaches with special emphasis on human language. Previously, Andrew was a researcher and analytics product manager at Digital Reasoning where he trained the world's largest artificial neural network with over 160 billion parameters, and helped guide the analytics roadmap for the Synthesys cognitive computing platform which tackles some of the most complex analysis tasks across government intelligence, finance, and healthcare industries.

Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
  • MEAP combo $49.99 pBook + PDF
  • MEAP eBook $39.99 PDF only

FREE domestic shipping on three or more pBooks