Three-Project Series

BERT-Based Transformer Projects you own this product

intermediate Python • intermediate PyTorch • basics of Natural Language Processing • basics of Google Colab
skills learned
loading and preprocessing a text data set • tokenizing data using pretrained tokenizers • loading and configuring pretrained ALBERT, RoBERTa, and DistilBERT models using Hugging Face
Rohan Khilnani
3 weeks · 6-8 hours per week average · INTERMEDIATE

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • share your subscription with another person
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases

lite $19.99 per month

  • access to all Manning books, including MEAPs!


5, 10 or 20 seats+ for your team - learn more

In this series of liveProjects, you’ll use variants of the BERT Transformer to solve real-world natural language processing problems. Transformers are pretrained machine learning models and are rapidly becoming the go-to architecture for any NLP use case. As you work through the hands-on challenges in this liveProject series, you’ll get real experience implementing state-of-the-art Transformer architectures, using the Hugging Face library, for use cases such as detecting hate speech, spotting fake news, and blocking spam. Each project is standalone, letting you pick and choose the application most relevant to your career. For each, you’ll create an entire NLP pipeline that stretches from preprocessing data all the way to validating model performance.

These projects are designed for learning purposes and are not complete, production-ready applications or solutions.

Manning author Rohan Khilnani shares what he likes about the Manning liveProject platform.

book resources

When you start each of the projects in this series, you'll get full access to the following book for 90 days.

choose your plan


only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • BERT-Based Transformer Projects project for free

project author

Rohan Khilnani
Rohan Khilnani is a data scientist at Optum, United Health Group. He has filed two patents in the field of natural language processing and has also published a research paper on LSTMs with Attention at the COLING conference in 2018.


This liveProject is for intermediate Python and NLP practitioners who are interested in implementing pretrained BERT architectures, and customizing them to solve real-world NLP problems. To begin this liveProject you will need to be familiar with the following:

  • Intermediate Python
  • Intermediate PyTorch
  • Basics of Google Colab
  • Basics of machine learning
  • Basics of neural networks
  • Basics of natural language processing

you will learn

In this liveProject, you will develop hands-on experience in building a text classifier using PyTorch Lightning and Hugging Face. You’ll also get practical experience working on GPUs in the Google Colab environment.

  • Working with Jupyter Notebook on Google Colab
  • Loading and preprocessing a text data set
  • Tokenizing data using pretrained tokenizers
  • Creating dataloaders and tensor data sets
  • Loading and configuring pretrained ALBERT model using Hugging Face
  • Building and training a text classifier using PyTorch Lightning
  • Validating the performance of the model by optimizing its precision score


You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from other participants and our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.