In this liveProject, you’ll step into the boots of an investigator trying to find the anonymous author of a seriously defamatory blog post. You’ve narrowed down your list of suspects, acquired a dataset of writing samples, and now plan to find the culprit using a custom machine learning project. Your challenge is to build an authorship analysis model that will match a sample to the defamatory blogpost and reveal the guilty party. To do this, you’ll need to extract data from a corpus of documents, build a model that can learn authorship style, scale the model to handle hundreds of suspects, and finally develop a user-friendly program that will allow non-technical colleagues to make use of your findings.
This project is designed for learning purposes and is not a complete, production-ready application or solution.
This liveProject is for software developers with an interest in data science, and beginner data scientists. It will require a machine with a minimum of 2GB of free harddrive space and 4GB of RAM. To begin this liveProject, you will need to be familiar with:
- Beginner Python and its utility functions
- Basics of pandas
- Basics of NumPy
- Basics of scikit-learn
- Basics of data science and machine learning
- Reading text files with Python
- Saving and loading Pandas DataFrames
- Running a training and evaluation experiment
- Running Python code from Jupyter Notebooks
- Basics of running terminal commands
you will learn
This liveProject will teach you important text mining and machine learning techniques that can be used for both author identification and other text-based tasks. The skills you’ll develop can also be applied to spotting bots on social media, building predictive models, and detecting fraud in electronic communications.
- Extract data from original datasets using XML parsing and BeautifulSoup
- Clean data to remove noise that will affect your model
- Extract useful features using preprocessing techniques, spaCy and scikit-learn
- Build a predictive classification model with scikit-learn
- Visualise authorship styles with an interactive plot built with altair
- Incorporate your trained model into a user-friendly program by converting Jupyter notebooks with Viola