Grokking Deep Reinforcement Learning
Miguel Morales
  • MEAP began May 2018
  • Publication in Early 2020 (estimated)
  • ISBN 9781617295454
  • 450 pages (estimated)
  • printed in black & white

The must-have book, for anyone that wants to have a profound understanding of deep reinforcement learning.

Julien Pohie

We all learn through trial and error. We avoid the things that cause us to experience pain and failure. We embrace and build on the things that give us reward and success. This common pattern is the foundation of deep reinforcement learning: building machine learning systems that explore and learn based on the responses of the environment.

Grokking Deep Reinforcement Learning introduces this powerful machine learning approach, using examples, illustrations, exercises, and crystal-clear teaching. You'll love the perfectly paced teaching and the clever, engaging writing style as you dig into this awesome exploration of reinforcement learning fundamentals, effective deep learning techniques, and practical applications in this emerging field.

Table of Contents detailed table of contents

Part 1: Reinforcement Learning Foundations

1. Introduction to Deep Reinforcement Learning

1.1. What is Deep Reinforcement Learning?

1.2. Why now?

1.3. When to use Deep Reinforcement Learning?

1.4. What will you learn in this book?

1.5. What do you need?

1.6. Summary

2. Planning For Sequential Decision-Making Problems

2.1. Architecture of a sequential decision-making problem

2.2. Objective of a decision-making agent

2.3. Planning optimal sequences of actions

2.4. Summary

3. Learning to Act Through Interaction

3.1. The challenge of interpreting evaluative feedback

3.2. Learning to estimate policies

3.3. Learning to optimize behavior

3.4. Summary

4. More Effective and Efficient Reinforcement Learning

4.1. Strategic exploration

4.2. Agents that that learn often and accurately

4.3. Agents that interact, learn and plan

4.4. Summary

Part 2: Deep Reinforcement Learning Algorithms

5. Introduction to value-based deep reinforcement learning

5.1. The kind of feedback a deep reinforcement learningagent deals with

5.2. Introduction to value-function approximation

5.3. NFQ: A first attempt to value-based deep reinforcementlearning

5.4. Summary

6. Stabilizing value-based deep reinforcement learning method

6.1. DQN: Making reinforcement learning more like supervised learning

6.2. Double DQN: Mitigating the overestimation of approximateaction-value functions

6.3. Summary

7. Sample-efficient value-based: deep reinforcement learning methods

7.1. Dueling DDQN: A reinforcement-learning-aware neural network architecture

7.1.1. Reinforcement learning is not a supervised learning problem

7.1.2. Value-based deep reinforcement learning methods nuances

7.1.3. Advantage of using advantages

7.1.4. A reinforcement-learning-aware architecture

7.1.5. Building a dueling network

7.1.6. Reconstructing the action-value function

7.1.7. Continuously updating the target network

7.1.8. What does the dueling network bring to the table?

7.2. PER: Prioritizing the replay of importantexperiences

7.2.1. A smarter way to replay experiences

7.2.2. Then, what is a good measure of "important" experiences?

7.2.3. Sampling by TD error

7.2.4. Prioritizing errors stochastically

7.2.5. Proportional prioritization

7.2.6. Rank-based prioritization

7.2.7. Prioritization bias

7.3. Summary

8. Gradient-Free Methods

Part 3: Advanced Applications

9. Advanced Exploration Strategies

10. Reinforcement Learning in Robots

11. Reinforcement Learning with Multiple Agents

12. Towards Artificial General Intelligence

About the Technology

Deep reinforcement learning is a form of machine learning in which AI agents learn optimal behavior on their own from raw sensory input. The system perceives the environment, interprets the results of its past decisions and uses this information to optimize its behavior for maximum long-term return. It has been said that deep reinforcement learning, which is the use of deep learning and reinforcement learning techniques to solve problems decision-making problems, is the solution to the full artificial intelligence problem.

Deep reinforcement learning famously contributed to the success of AlphaGo and all its successors (AlphaGo, AlphaGo Zero and AlphaZero, etc), which recently beat the world’s best human player in the world’s most difficult board game. But, that is not the only thing you can do with deep reinforcement learning. These are some of the most notable applications:

Learn to play ATARI games just by looking at the raw image.
Learn to trade and manage portfolios effectively.
Learn low-level control policies for a variety of real-world models.
Discover tactics and collaborative behavior for improved campaign performance.
From low-level control, to high-level tactical actions, deep reinforcement learning can solve large, complex decision-making problems.

But, deep reinforcement learning is an emerging approach, so the best ideas are still yours to discover. We can’t wait to see how you apply deep reinforcement learning to solve some of the most challenging problems in the world.

About the book

Grokking Deep Reinforcement Learning is a beautifully balanced approach to teaching, offering numerous large and small examples, annotated diagrams and code, engaging exercises, and skillfully crafted writing. You'll explore, discover, and learn as you lock in the ins and outs of reinforcement learning, neural networks, and AI agents. You will go from small grid world environments and some of the foundational algorithms to some of the most challenging environments out there today and cutting-edge techniques to solve these environments.

Exciting, fun, and maybe even a little dangerous. Let's get started!

What's inside

  • Foundational reinforcement learning concepts and methods
  • The most popular deep reinforcement learning agents solving high-dimensional environments
  • Cutting-edge agents that emulate human-like behavior and techniques for artificial general intelligence

About the reader

Written for developers with some understanding of deep learning algorithms. Experience with reinforcement learning is not required. Perfect for readers of Deep Learning in Python or Grokking Deep Learning.

About the author

Miguel Morales is a Senior Software Engineer at Lockheed Martin, Missile and Fire Control-Autonomous Systems. He is also a faculty member at Georgia Institute of Technology where he works as an Instructional Associate for the Reinforcement Learning and Decision Making graduate course. Miguel has worked for numerous other educational and technology companies including Udacity, AT&T, Cisco, and HPE.

Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
MEAP combo
$35.00 $49.99 pBook + eBook + liveBook
MEAP eBook
$25.00 $39.99 pdf + ePub + kindle + liveBook

placing your order...

Don't refresh or navigate away from the page.

FREE domestic shipping on three or more pBooks