Welcome to Manning India!

We are pleased to be able to offer regional eBook pricing for Indian residents.
All eBook prices are discounted 40% or more!
Transfer Learning for Natural Language Processing
Paul Azunre
  • MEAP began March 2020
  • Publication in Fall 2020 (estimated)
  • ISBN 9781617297267
  • 325 pages (estimated)
  • printed in black & white

A complex topic is broken down into manageable pieces, while maintaining a good pace. The text is accompanied by replicable code examples throughout.

Mathijs Affourtit
Building and training deep learning models from scratch is costly, time-consuming, and requires massive amounts of data. To address this concern, cutting-edge transfer learning techniques enable you to start with pretrained models you can tweak to meet your exact needs. In Transfer Learning for Natural Language Processing, DARPA researcher Paul Azunre takes you hands-on with customizing these open source resources for your own NLP architectures. You’ll learn how to use transfer learning to deliver state-of-the-art results even when working with limited label data, all while saving on training time and computational costs.
Table of Contents detailed table of contents

Part 1: What is Transfer Learning?

1 What is Transfer Learning

1.1 Overview of representative NLP tasks

1.2 Understanding Natural Language Processing (NLP) in the Context of Artificial Intelligence (AI)

1.3 A Brief History of NLP Advances

1.4 Why is NLP Transfer Learning Important Now?

1.5 Summary

2 Getting Started with Baselines

2.1 Preprocessing Email Spam Classification Example Data

2.2 Preprocessing Movie Sentiment Classification Example Data

2.3 Generalized Linear Models

2.4 Decision-Tree-Based Models

2.5 Neural-Network-Based Models

2.6 Optimizing Performance

2.7 Summary

Part 2: Transfer Learning for NLP

3 Shallow Transfer Learning in NLP

3.1 Semi-supervised Learning with Word Embeddings

3.2 Semi-supervised Learning with Higher-level Representations

3.3 Multi-task Learning

3.4 Domain Adaptation

3.5 Summary

4 Deep Transfer Learning in NLP with CNNs and RNNs

4.1 Introducing Fact Checking and Data Type Classification Examples

4.2 Transferability of CNN Layers

4.3 Embeddings from Language Models (ELMo)

4.4 Semantic Inference for the Modeling of Ontologies (SIMOn)

4.5 Universal Language Model Fine-tuning for Text Classification (ULM-FiT)

4.6 Summary

5 Deep Transfer Learning in NLP with Transformers

5.1 The Transformer

5.2 The OpenAI Transformer

5.3 Bidirectional Encoder Representations from Transformers (BERT)

5.4 Revisiting Fact Checking Example

5.5 Revisiting Data Type Classification Example

5.6 Cross-lingual Learning with Multilingual BERT (mBERT)

5.7 Summary

6 Deep Transfer Learning Adaptation Strategies

6.1 Gradual Unfreezing and Sequential Adaptation

6.2 Discriminative Fine-tuning

6.3 Weak Supervision and Multitask Fine-tuning

6.4 Ensembling and Distilling

6.5 Achieving Greater Parameter Efficiency via Adapters

6.6 Summary

7 Conclusions [25 pages]

7.1 Key concepts review

7.2 Advice on picking the right Transfer Learning approach

7.3 Discussion of possible limitations of models

7.4 Immediate future of Transfer Learning for NLP

7.5 Long-term future of Transfer Learning for NLP

7.6 Final Words on Staying Up-to-date in the Field


Appendix A - Free GPUs in the Cloud with Kaggle Kernels and Colab Notebooks

Appendix B - Using Microservices to Handle Dependencies

About the Technology

Transfer learning enables machine learning models to be initialized with existing prior knowledge. Initially pioneered in computer vision, transfer learning techniques have been revolutionising Natural Language Processing with big reductions in the training time and computation power needed for a model to start delivering results. Emerging pretrained language models such as ELMo and BERT have opened up new possibilities for NLP developers working in machine translation, semantic analysis, business analytics, and natural language generation.

About the book

Transfer Learning for Natural Language Processing is a practical primer to transfer learning techniques capable of delivering huge improvements to your NLP models. Written by DARPA researcher Paul Azunre, this practical book gets you up to speed with the relevant ML concepts before diving into the cutting-edge advances that are defining the future of NLP. You’ll learn how to adapt existing state-of-the art models into real-world applications, including building a spam email classifier, a movie review sentiment analyzer, an automated fact checker, a question-answering system and a translation system for low-resource languages. For each NLP application, you’ll learn how to setup a microservices software architecture that will fine-tune your model as new data comes in.

What's inside

  • Fine tuning pretrained models with new domain data
  • Picking the right model to reduce resource usage
  • Transfer learning for neural network architectures
  • Foundations for exploring NLP academic literature

About the reader

For machine learning engineers and data scientists with some experience in NLP.

About the author

Paul Azunre holds a PhD in Computer Science from MIT. He works as a Research Director studying Transfer Learning in Natural Language Processing, and has pushed the state of the field with peer-reviewed articles, serving as a program committee member, speaking engagements, and judging at top conferences. Paul has also served as a Principal Investigator on several DARPA research programs at the US Department of Defense.

Manning Early Access Program (MEAP) Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
MEAP combo $49.99 pBook + eBook + liveBook
MEAP eBook $39.99 pdf + ePub + kindle + liveBook
Prices displayed in rupees will be charged in USD when you check out.

placing your order...

Don't refresh or navigate away from the page.

FREE domestic shipping on three or more pBooks