In this liveProject, you’ll use the RoBERTa variation of the BERT Transformer to detect occurrences of fake news in a data set. Political news can be tricky to validate for accuracy, as sources report the same events from different biased angles. RoBERTa uses different pre-training methods than traditional BERT and has hyperparameters that are highly optimized, meaning it tends to perform better than its predecessor. You’ll start out by loading the model using the Hugging Face library and training it to your data with PyTorch Lightning. The project will also include the implementation of training a custom tokenizer from scratch and using it to tokenize the data. A successful model will maximize positives, and so model evaluation will be based on your model having a high recall score.
This project is designed for learning purposes and is not a complete, production-ready application or solution.