FREE
You can see any available part of this book for free.Click the table of contents to start reading.
Building and training deep learning models from scratch is costly, time-consuming, and requires massive amounts of data. To address this concern, cutting-edge transfer learning techniques enable you to start with pretrained models you can tweak to meet your exact needs. In Transfer Learning for Natural Language Processing, DARPA researcher Paul Azunre takes you hands-on with customizing these open source resources for your own NLP architectures. You’ll learn how to use transfer learning to deliver state-of-the-art results even when working with limited label data, all while saving on training time and computational costs.A complex topic is broken down into manageable pieces, while maintaining a good pace. The text is accompanied by replicable code examples throughout.
About the Technology
Transfer learning enables machine learning models to be initialized with existing prior knowledge. Initially pioneered in computer vision, transfer learning techniques have been revolutionising Natural Language Processing with big reductions in the training time and computation power needed for a model to start delivering results. Emerging pretrained language models such as ELMo and BERT have opened up new possibilities for NLP developers working in machine translation, semantic analysis, business analytics, and natural language generation.About the book
Transfer Learning for Natural Language Processing is a practical primer to transfer learning techniques capable of delivering huge improvements to your NLP models. Written by DARPA researcher Paul Azunre, this practical book gets you up to speed with the relevant ML concepts before diving into the cutting-edge advances that are defining the future of NLP. You’ll learn how to adapt existing state-of-the art models into real-world applications, including building a spam email classifier, a movie review sentiment analyzer, an automated fact checker, a question-answering system and a translation system for low-resource languages. For each NLP application, you’ll learn how to setup a microservices software architecture that will fine-tune your model as new data comes in.
Table of Contents takes you straight to the bookdetailed table of contents
Part 1: Introduction and Overview
1 What is Transfer Learning
1.1 Overview of Representative NLP Tasks
1.2 Understanding Natural Language Processing (NLP) in the Context of Artificial Intelligence (AI)
1.2.1 Artificial Intelligence (AI)
1.2.2 Machine Learning
1.2.3 Natural Language Processing (NLP)
1.3 A Brief History of NLP Advances
1.3.1 General Overview
1.3.2 Recent Transfer Learning Advances
1.4 Transfer Learning in Computer Vision
1.4.1 General Overview
1.4.2 Pre-trained ImageNet Models
1.4.3 Fine-tuning Pre-trained ImageNet Models
1.5 Why is NLP Transfer Learning an Exciting Topic to Study Now?
1.6 Summary
2 Getting Started with Baselines
2.1 Preprocessing Email Spam Classification Example Data
2.1.1 Loading and Visualizing the Enron Corpus
2.1.2 Loading and Visualizing the Fraudulent Email Corpus
2.1.3 Converting the Email Text Into Numbers
2.2 Preprocessing Movie Sentiment Classification Example Data
2.3 Generalized Linear Models
2.3.1 Logistic Regression
2.3.2 Support Vector Machines (SVMs)
2.4 Decision-Tree-Based Models
2.4.1 Random Forests (RFs)
2.4.2 Gradient Boosting Machines (GBMs)
2.5 Neural Network Models
2.5.1 Embeddings from Language Models (ELMo)
2.5.2 Bidirectional Encoder Representations from Transformers (BERT)
2.6 Optimizing Performance
2.6.1 Manual Hyperparameter Tuning
2.6.2 Systematic Hyperparameter Tuning
2.7 Summary
Part 2: Transfer Learning for NLP
3 Shallow Transfer Learning for NLP
3.1 Semi-supervised Learning with Pretrained Word Embeddings
3.2 Semi-supervised Learning with Higher-Level Representations
3.3 Multi-Task Learning
3.3.1 Problem Setup and Shallow Neural Single-Task Baseline
3.3.2 Dual Task Experiment
3.4 Domain Adaptation
3.5 Summary
4 Deep Transfer Learning for NLP with Recurrent Neural Networks
4.1 Preprocessing Tabular Column Type Classification Data
4.1.1 Obtaining and Visualizing Tabular Data
4.1.2 Preprocessing Tabular Data
4.1.3 Encoding Preprocessed Data as Numbers
4.2 Preprocessing Fact Checking Example Data
4.2.1 Special Problem Considerations
4.2.2 Loading and Visualizing Fact-Checking Data
4.3 Semantic Inference for the Modeling of Ontologies (SIMOn)
4.3.1 General Neural Architecture Overview
4.3.2 Modeling Tabular Data
4.3.3 Application of SIMOn to tabular Column Type Classification Data
4.4 Embeddings from Language Models (ELMo)
4.4.1 ELMo Bidirectional Language Modeling
4.4.2 Application to Fake News Detection
4.5 Universal Language Model Fine-Tuning (ULMFiT)
4.5.1 Target Task Language Model Fine-Tuning
4.5.2 Target Task Classifier Fine-Tuning
4.6 Summary
5 Deep Transfer Learning in NLP with Transformers
6 Deep Transfer Learning Adaptation Strategies
7 Conclusions
Appendixes
Free GPUs in the Cloud with Kaggle Kernels and Colab Notebooks
Using Microservices to Handle Dependencies
What's inside
- Fine tuning pretrained models with new domain data
- Picking the right model to reduce resource usage
- Transfer learning for neural network architectures
- Foundations for exploring NLP academic literature
About the author
Paul Azunre holds a PhD in Computer Science from MIT. He works as a Research Director studying Transfer Learning in Natural Language Processing, and has pushed the state of the field with peer-reviewed articles, serving as a program committee member, speaking engagements, and judging at top conferences. Paul has also served as a Principal Investigator on several DARPA research programs at the US Department of Defense.placing your order...
Don't refresh or navigate away from the page.
Manning Early Access Program (MEAP)
Read chapters as they are written, get the finished eBook as soon as it’s ready, and receive the pBook long before it's in bookstores.
Transfer Learning for Natural Language Processing (print book) added to cart
continue shopping
go to cart
Transfer Learning for Natural Language Processing (eBook) added to cart
continue shopping
go to cart
Prices displayed in rupees will be charged in USD when you check out.
FREE domestic shipping on three or more pBooks