20 Questions

In this liveProject, you’ll use Python to create a command-line version of the classic guessing game 20 Questions. In your version of the game, the computer will have up to twenty chances to guess what animal the player is thinking of. You’ll use Jupyter notebooks to code your program and define functions that build a knowledge base in real time. Then, you’ll test and debug your code by playing against the computer.

skills learned: data processing • user input • randomness • data filtering • CRUD operations • modularizing code • debugging

2D Plotting with Matplotlib

ABC Bikes Inc. is considering launching a bike rental service in your area. As the company’s data analyst, your task is to provide decision-driving insights on bike rental trends. You’ll extract the relevant data from a publicly available dataset into a pandas data frame. Then, using Python plotting library Matplotlib, you’ll create line plots to visualize changes for bike rentals on a single day as well as over a specific time period, create a grouped bar plot to visualize a comparison of renting data for two years, create subplots with different chart types, and create a violin plot to visualize renting patterns over four seasons. When you’re done, you’ll know how to use Matplotlib to create accurate and informative 2D visualizations.

skills learned: use import statement to load required libraries and modules into Jupyter Notebook • use pandas functions to read data from CSV files into a data frame object • use pandas functions to subset required rows and columns for plotting • use Matplotlib API/functions to create, customize, and save plots

3D Medical Image Analysis with PyTorch

In this liveProject, you’ll take on the role of a machine learning engineer at a healthcare imaging company, processing and analyzing magnetic resonance (MR) brain images. Your current medical image analysis pipelines are set up to use two types of MR images, but a new set of customer data has only one of those types! Your challenge is to build a convolutional neural network that can perform an image translation to provide you with your missing data. You’ll do this using the deep learning framework PyTorch and a large preprocessed set of MR brain images. The company also wants to make sure your image translation convolutional neural network reliably produces the desired MR image, so you will need to provide qualitative and quantitative results demonstrating your method’s effectiveness.

skills learned: train a neural network for a regression task • build a CNN • handle and visualize medical imaging data

3D Plotting with Matplotlib

ABC Bikes Inc. is considering expanding its bike rental service to a new county. Your job, as the company’s data analyst, is to determine factors that impact the demand for rental bikes in that area. You’ll extract the relevant data from a publicly available dataset into a pandas data frame. Then, using Matplotlib’s mplot3d toolkit, you’ll plot 3D graphs to simultaneously visualize more than two data features, enabling you to determine useful patterns including how temperature, time of day, and month of year impact bike rentals. When you’re finished, you’ll know how to use Matplotlib to create, customize, and rotate your 3D plots to gather useful and interesting insights.

skills learned: visualize on 3D scatter plot and 3D surface plot • interactive visualization

Add Background to a Green Screen Video

In this quick liveProject, you’ll develop a low-resource way to add a background to a green screen video using thresholding. You’ll read the frames of a green screen video, perform thresholding to separate the green screen background, apply the new background image, and write the revised frames to a new video.

skills learned: reading and writing videos • using binary thresholding

Add Database Functionality

Logistica Transport needs to ditch its old desktop app and embrace a new web app solution. They’ve turned to you to help them, and in this liveProject, you’ll utilize Entity Framework Core to set up the vital connections between a MySQL database and the future web app. You’ll need to set up data models that define the structure and relationships of your data, as well as learn how to perform CRUD operations to create, update, and delete data in your database. Finally, you’ll connect your data models and CRUD operations to the Entity Framework for a full and working database solution.

skills learned: NuGet packages specific to Entity Framework and MySQL • Setting up data classes in a database • Data tables (inserting, updating, and deleting information) • Setting up a database connection in code • Working with data stored in database tables

Adversarial Training

Protect your model by implementing adversarial training, the easiest method of safeguarding against adversarial attacks. You’ll load your dataset, learn its structure, and examine a few random samples using OpenCV or Matplotlib. Using Numpy, you’ll prepare your dataset for training, then you’ll use FGSM to generate malicious input for both untargeted and targeted attacks on a trained DL model. For each type of attack, you’ll evaluate your model before and after you apply adversarial training-based mitigation methods, gauging the success of your defense.

skills learned: basics of adversarial training · iterative retraining with Keras • CleverHans attack generator

Algorithmic Trading with Machine Learning

Machine learning and the growing availability of diverse financial data has created powerful and exciting new approaches to quantitative investment. In this liveProject, you’ll step into the role of a data scientist for a hedge fund to deliver a machine learning model that can inform a profitable trading strategy.

You’ll go hands-on to build an end-to-end strategy workflow that includes sourcing market data, engineering predictive features, and designing and comparing various ML models. Throughout the liveProject you will work with libraries and tools from the industry-standard Python data ecosystem. You’ll tackle challenges such as training a regularized linear regression model, tuning a gradient boosting ML model, and evaluating the performance of your strategy—all essential skills for success in this highly lucrative area of machine learning.

skills learned: engineer financial features • build and test fundamental and advanced ML models to predict asset returns with scikit-learn and LightGBM • develop a trading strategy by defining rules that translate model predictions into trades

Analysis and Feature Extraction

Feature extraction is an essential part of detecting deepfake videos. In this liveProject, you’ll analyze both deepfaked and real video data in order to determine what features are common in faked videos. You’ll then compute those features for faces detected in the videos to determine which are fake and which are real.

skills learned: understand the differences between deepfake faces and real faces • compare images and their histograms

Analyze Reports with Hugging Face

You’re a data scientist at Finative, an environmental, social, and governance (ESG) analytics company that analyzes a high volume of data using advanced natural language processing (NLP) techniques in order to provide its clients insights for sustainable investing. Recently, your CEO has decided that Finative should increase its own financial sustainability. Your task is to classify sustainability reports of a publicly traded company in an efficient and sustainable way.

You’ll learn the fundamental mathematics—including backpropagation, matrix multiplication, and attention mechanisms—of Transformers, empowering you to optimize your model’s performance, improve its efficiency, and handle undesirable model predictions. You’ll use Python’s pdfplumber library to extract text from a sustainability report for quick delivery to your CEO. To further increase efficiency, you’ll save training time by using a language model that’s been pre-trained with ESG data to build a pipeline for the model and classify the sustainability report.

skills learned: understand the attention mechanism (Transformers) • build a model pipeline with Hugging Face • extract and prepare data from PDF files

Analyze a Trust Network

In this liveProject, you’ll turn your data science skills to analyzing an OTC network dataset scraped from bitcoin users in order to establish the most (and least!) trustworthy users. You’ll analyze a provided graph dataset, visualize it, generate features, and then create user clusters. You’ll start out by reading and examining the trust network dataset in Python, then create and interpret user clusters, and finally visualize the nodes and edges of the network dataset. This fast and engaging data science project will stretch your skills and build your knowledge of clustering.

skills learned: pandas for data operations • scikit-learn for clustering • NetworkX to analyze and visualize network datasets

Analyze and Cluster OpenStreetMap Data

In this liveProject, you’ll use LynxKite and graph data techniques to identify some of the most important geographic points in the city of Bruges. You’ll start by downloading and processing map data, and then use a simple Python program to convert it into a graph. You’ll use graph centrality metrics to quantify the importance of vertices in your graph, and determine some of Bruges’ important locations. You’ll then use the same structure to figure out the main areas of the city without using actual district data.

skills learned: convert map data to a graph representation • visualize graphs • compute and use common centrality metrics

Analyze the Graph Structure of Soccer

In this liveProject, you’ll construct event sequence graphs to reveal interesting information about soccer games. You’ll work to find longest pass sequences, most important players, and to understand the spatial structure of the game. You’ll define and visualize these graphs, and use connected components to find interesting event subsequences. You’ll quickly be able to uncover insights such as the most important players and the spatial structuring of the playing pitch.

skills learned: defining graphs based on event sequences • defining a pass graph among players • defining a graph on areas of the pitch

Analyzing GitHub Data with GCP Services

GitHub is stuffed full of free and open-source development tools, all with no guarantee that they’re stable, up-to-date, or even still maintained! Companies need a way to ensure that the GitHub projects they’re adopting come with a clean bill of health — and for that, they turn to you!

You’re the CTO of GitHub Health, a unique startup that provides reports and analysis of GitHub software projects. But your findings aren’t based on gossip, they’re based on data. In this liveProject, you’ll build a serverless data system that can extract meaningful data from GitHub, store it in a database, and display the statistics using Google Cloud. Once you’re done, you’ll generate a report for a new client on which of three open-source projects is the best choice for its new DevOps team.

skills learned: serverless data extraction • storage and presentation within Google Cloud

Apply a Face Mask to a Selfie

In this quick liveProject, you’ll develop a selfie enhancement application to build awareness of the importance of mask-wearing during a pandemic. You’ll use a laptop webcam to capture a user’s selfie, load a face mask image with a transparent background, and apply the face mask onto the image at the location selected by the user.

skills learned: tracking mouse events • using bitwise operations • saving images

Arbitrage Pricing Model

In this liveProject, you’ll build a machine learning system based on the Arbitrage Pricing Theorem (APT) that can create a diversified investment portfolio designed to avoid risks. APT allows you to model the effects of different scenarios on an investment portfolio, and you’ll test this theorem on a portfolio of social media companies. You’ll run analysis to estimate coefficients, and conduct sensitivity analysis to determine important macroeconomic factors. Finally, you’ll interpret your results to find out what issues your portfolio is the most sensitive to.

skills learned: risk management • multi-factor modeling • multivariate linear regression

Augment Training Data and Classify Text

In this liveProject, you’ll augment text-based training data for a sentiment analysis algorithm with artificially generated positive reviews. You’ll merge the synthetic positive reviews with an unbalanced dataset focused on negative reviews, thereby creating a balanced dataset for your model to train on. You’ll train your model, then evaluate its metrics using sklearn.

skills learned: merging training and synthetic data • building and training a text classification model • scoring text data with the model

Authorship Identification with Text Mining and Machine Learning

In this liveProject, you’ll step into the boots of an investigator trying to find the anonymous author of a seriously defamatory blog post. You’ve narrowed down your list of suspects, acquired a dataset of writing samples, and now plan to find the culprit using a custom machine learning project. Your challenge is to build an authorship analysis model that will match a sample to the defamatory blogpost and reveal the guilty party. To do this, you’ll need to extract data from a corpus of documents, build a model that can learn authorship style, scale the model to handle hundreds of suspects, and finally develop a user-friendly program that will allow non-technical colleagues to make use of your findings.

skills learned: extract features from text using scikit-learn and spaCy • build a predictive classification model • visualize authorship styles with an interactive plot • incorporate your trained model into a user-friendly program

Automate Data Preprocessing

In this liveProject, you’ll utilize automated machine learning tools to help the data preprocessing and feature engineering for creating an image classification model. You’ll start with the basics of data preprocessing, and then see how useful AutoML solutions can make this process quicker and easier. The rewards are big, as properly preprocessed data can dramatically improve the outcomes of a deep learning project.

skills learned: implementing data preprocessing for image data • training deep learning models adopting the data preprocessing

Automate Pipeline

In this liveProject, you’ll implement an AutoML pipeline using the AutoKeras functional API. You’ll make use of the built-in blocks in AutoKeras to conduct automated hyperparameter tuning and model selection for creating an image classification model. Your challenges will include customizing both a sequential AutoML pipeline, and customizing graph-structured AutoML Pipeline.

skills learned: implementing AutoML using the automated data preprocessing API and AutoKeras Task API • customizing sequential and graph-structured AutoML pipelines

Basic API of AutoKeras

In this liveProject, you’ll learn how to use the Task and Functional API of Keras to build an automated deep learning model for image classification. This AutoML approach will allow you to avoid time spent selecting and tuning your deep learning algorithms. You’ll work with the detailed CIFAR dataset of animal images which is easy to access through the TensorFlow Keras API.

skills learned: task and Functional API of AutoKeras • training deep learning models automatically with AutoKeras

Basic Deep Learning

In this liveProject, you’ll learn and apply some of the basics of deep learning in order to build the foundations of an AutoML image classifier. You’ll discover the basic deep learning models for image classification, and then experiment with tuning the hyperparameters of a convolutional neural network to improve your results.

skills learned: constructing deep learning models for image classification • search the hyperparameters of deep models with grid search and random search

Bayesian Dynamic Linear Modeling

In this liveProject, you’ll build a Bayesian dynamic linear model that can take account of sudden state space changes and rapidly react to dramatic trend changes. These trend changes could take many forms—from heightened demand during a major sporting event, to a global pandemic that causes cancellations to skyrocket. You’ll use the PyDLM library to generate forecasts that can dynamically adapt to the unforeseen, and quickly shift to making accurate predictions for a new reality.

skills learned: build a simple DLM model • build Bayesian dynamic linear models with the PyDLM library

Bayesian Statistical Methods with PyMC3

In this liveProject, you’ll use PyMC3 to generate a posterior distribution of hotel cancellations. This will allow you to build a predictive model that can update its predicted probabilities by incorporating new information. You’ll master methods for modelling mean and standard deviations based on the selected prior values, generating distributions to determine if data follows an AR(1) process, modeling of stochastic volatility and generalized linear modelling with PyMC3.

skills learned: manipulating a dataset with pandas • probabilistic time series analysis with PyMC3 • generating posterior distributions • modeling autoregressive processes

Boost Specific Results

Help Sneakers to the Max’s customers find the perfect sneakers! As a software engineer at the e-commerce company, your task is to improve its search engine—and hopefully sales—by fine-tuning the order of search results. You’ll adjust the index mapping to import data about sneaker release dates, popularity, and in-stock status. Then, you’ll write extensions to the Elasticsearch queries that boost recent and popular results, forcing out-of-stock sneakers lower. Finally, you’ll use everything you learned to return the most relevant sneakers using a judgment list provided by the product managers. When you’re finished, you’ll have the skills to enhance Elasticsearch queries, while Sneakers to the Max’s customers will easily find the perfect pair.

skills learned: analyze the results of different boosting mechanisms and tune them to work together

Break Strings into Substrings

You’re a PowerShell developer working for XYZ Services Company, a consulting and outsourcing firm that recently experienced a massive failure in its IT infrastructure. While it’s working on getting the environment back up and running, you’ve been given the critical, time-sensitive task of helping avoid payroll delays. The good news is the company’s employee directory has already been extracted. The bad news is it’s been saved as plain text files, a hard-to-work-with format for payroll purposes. Applying PowerShell best practices, you’ll write scripts that parse text data and convert it into objects that are in a payroll-manageable format.

skills learned: read text files • use delimiter characters • basic regex • extract substrings from text • arrays and array indexes • create custom objects • add properties to objects

Build Network Models for Pandemics

In this liveProject, you’ll use machine learning to construct a contact tracing network for COVID-19 using location recordings from smart phone data. You’ll read the location of infected individuals, and generate a contact network of individuals who have been within two meters. Once you’ve established this tracing system, you’ll implement a distributed algorithm that can compute the average infection rate for each connected component of the contact network.

skills learned: reading in location recordings from a CSV file • determining geodesic distances between locations • implement a simple algorithm that computes a summary statistic of networked data

Build a Data Pipeline with BigQuery

In this liveProject, you’ll create a data pipeline that can enrich source data with currency exchange rates and converted totals. As this pipeline is updated daily, you’ll need to create a scheduled query to enrich your data. You’ll learn to load essential data into BigQuery from both CSVS and JSON, and use it to generate a daily business intelligence report for your colleagues.

skills learned: creating tables from CSV and nested JSON data • scheduling your queries and dataset updates • creating materialized views and aggregated activity tables

Build a ResNet Model

In this liveProject, you’ll build a ResNet deep learning model from scratch to analyze medical imagery. A ResNet is a deep neural network model which uses "Residual blocks" and "skip connections" to reduce the need for very deep networks while still achieving high accuracy. You’ll then train your model on X-ray and CT datasets, and plot validation loss, and accuracies vs. epochs. You’ll build an important familiarity with the functional blocks of a DL model, how data must be formatted, and which layers to use to solve your problems.

skills learned: build a ResNet deep learning architecture with basic functional components in Keras • train ResNet model hyperparameters on two different types of medical image datasets (X-ray, CT) • tune ResNet model to improve performance

Build a Self-Serve Data Platform

You’re a consultant working for Messflix Inc., a movie and TV-show streaming platform. Your task is to set up a Python prototype implementation of the data mesh to roll out the technical components of a data mesh. Using Python and pandas, you’ll write a Python function that creates an empty CSV file with the predefined attributes of your data product, builds a data catalog by creating Python functions that write to the CSV file, and sets up standardized access to the CSV datasets. When you’re done, you’ll have hands-on experience building a minimal self-serve data platform using simple techniques.

skills learned: build a self-serve data platform • build a small data catalog • consume data inside a data mesh • store data products

Build a VGG16 Model

In this liveProject, you’ll build a VGG16 deep learning model from scratch to analyze medical imagery. A VGG16 is a deep convolutional network model which has shown to achieve high accuracy in image based pattern recognition tasks. You’ll then train your model on X-ray and CT datasets, and plot validation loss, and accuracies vs. epochs. You’ll build an important familiarity with the functional blocks of a DL model, how data must be formatted, and which layers to use to solve your problems.

skills learned: build a VGG16 deep learning architecture in Keras • use custom image data generators in Keras • train VGG16 model on two different types of medical image datasets (X-ray, CT) • tune VGG16 model hyperparameters to improve performance

Build a Web App

In this liveProject you’ll create a React web application and then integrate TensorFlow.js machine learning functionality into the app. React is highly prized by developers for its ease of building simple and intuitive frontends. Once you’ve set up your development environment and built your React-based UI, you’ll train your TensorFlow.js model with synthetic data.

skills learned: creating a React web app and analyzing app structure • creating a basic UI and event listeners in React • constructing a neural network model with TensorFlow.js

Build an ML Recommender System

Recommender systems are one of the most popular and lucrative uses of machine learning, allowing businesses and organizations to give personalized suggestions to their customers.

In this liveProject, you’ll use common tools of the Python data ecosystem to design, build, and evaluate a movie recommendation model for the movie website. You’ll kick off your project by building simple genre charts, then work with existing movie rating data to implement personalized recommendations for each customer. When you’ve completed this hands-on and interesting project, you’ll have mastered a cornerstone technique of machine learning that’s in demand across companies and industries.

skills learned: load and analyze a dataset • train a machine learning model • evaluate a machine learning model • use a trained ML model on a website • collaborative filtering

Building Domain-Specific Language Models

In this liveProject, you’ll step into the role of a natural language processing data scientist working for Stack Exchange. Stack Exchange runs a network of question-and-answer sites on diverse topics ranging from programming to cooking. Your boss wants you to create language models that are tuned to the statistical, probabilistic, and technical jargon present in different Stack Exchange sites.

Language is domain-specific—an insurance company’s documents will use very different terminology than a post on a social media site. Because of this, off-the-shelf NLP models trained on generic text can be inaccurate for specialized domains such as healthcare, legal, clinical, and agricultural language. Your goal is to build a language model capable of query completion and larger text generation for Stack Exchange sites. At the end of this project, you will be able to build the foundations of any domain-specific NLP system by creating a robust and efficient language model using statistical and deep learning techniques.

Updated: March 2022

• Fully updated to the latest version of AllenNLP
• Improved GPU compatibility for training larger models
• New help layers with detailed hints and guidance
• New preprocessing steps for data preparation
• Adjusted prerequisites and libraries

skills learned: data manipulation with NumPy and pandas • text preprocessing with NLTK • train an RNN with PyTorch • score and evaluate language models

Building a Search API with Elasticsearch and BERT

In this liveProject, you’ll bring together multiple tools and models to construct a production-grade smart search engine. You’ll combine off-the-shelf Elasticsearch models for keyword search with your own semantic search API using transformers. Start by setting up and indexing an Elasticsearch Docker container, then quickly move on to boosting search relevance with BERT. Finally, you’ll set up a Flask API to serve a BERT model and look into customizing your search engine for your own particular topics of interest.

skills learned: Packaging and deploying with Docker • search and indexing with Elasticsearch • building APIs with Flask

Building an ML Pipeline with Kubeflow

Putting machine learning into production can often be a complex task. The Kubeflow platform helps streamline this process with simple and scalable ML workflow deployment. In this liveProject, you’ll put Kubeflow into action to help your team roll out their new license plate recognition deep learning system.

You’ll help data scientist colleagues by standardizing their working environment, and automating away many tedious and error-prone tasks. Your challenges will include restructuring a complex deep learning project to make it Kubeflow-friendly, and developing reusable components that can be transferred to other machine learning pipelines.

skills learned: create a machine learning pipeline that is composable and scalable • structure a non-trivial ML project to make it Kubeflow-friendly • view training runs in Tensorboard • use Kubeflow Metadata to capture and locate generated data

Business Intelligence with BigQuery

In this liveProject, you’ll create a Google Data Studio report that can visualise your datasets for the purpose of business intelligence. You’ll connect up numerous data sources using BigQuery to feed directly into your new BI dashboard. You’ll design and implement revenue and daily spend metrics and line graphs, and scorecards for top users.

skills learned: connect a BigQuery table as a GDS data source • create a table chart in GDS • add bar charts and slider controls to a GDS dashboard

Capital Asset Pricing Model

In this liveProject, you’ll build a machine learning system based on the Capital Asset Pricing Model (CAPM) that can determine if certain stocks are over or undervalued, and the risk level of these stocks relative to the market index. CAPM is a powerful tool in finance due to its intuitive and easy-to-apply nature, and in this project you’ll use it to estimate coefficients, determine valuation accuracy, and finally find which stock promises the best risk-return relationship.

skills learned: risk management • valuation • univariate linear regression

Churn Analysis

In this liveProject, you’ll analyze a data set of customers metrics to uncover patterns and behaviors that indicate customer churn. You’ll determine key details like rate of retention, and analyze your data to determine which metrics are associated with customers leaving and customers staying loyal. You’ll analyze how customer behaviors are related to one another, and see if you can expand on your datasets functionality to find new metrics that indicate churn.

skills learned: create a data set from raw data stored in a database • visualize how different customer metrics relate to the outcome of churn • analyze different customer behaviors

Churn Forecasting

In this liveProject, you’ll develop a simple machine learning algorithm that can determine from data metrics which of your customers are likely to churn out of an online business. You’ll develop an XGBoost machine learning model and explain its predictions, then build a logistic regression statistical forecasting and analysis model to also predict churn. By the time you’re done, you’ll have two reliable and automated tools for spotting when customers are likely to leave so that your marketing team can easily intercede.

skills learned: XGBoost machine learning model churn forecasting • logistic regression model churn forecasting • explain machine learning model results

Class Methods and Variables

In this liveProject, you’ll write a Python class to manage network connections to your server. You want your class to track the port number and the connections, but instance data is, by definition, not associated with your class. Your challenge is to explore and determine the best ways to associate data with a class and learn how to use a class method.

skills learned: write a Python class to manage network connections • explore ways to access class variables • create class methods

Classification with XGBoost

In this liveProject, you’ll use the Julia language to build a classification-based machine learning model that can predict the salary of a customer based on their sociodemographic data. This model will then be used to serve premium advertising to wealthier customers. You’ll build and evaluate XGBoost models with the dedicated Julia XGBoost.jl package, tune the hyperparameters, and assess your model’s capabilities using ROC curve, and measures such as AUC, accuracy, recall, and precision.

skills learned: comparing distributions of predefined train and test data • building XGBoost model in Julia • evaluating classification model’s quality

Classify a Workout

In this liveProject, you’ll test the functionality of a TensforFlow.js based fitness assistant and assess its capabilities of pose estimation for workout sessions. You’ll move data from a PoseNet model to a TensorFlow.js inference function that can recognize a workout type and add delays to prevent unnecessary logging duplication. You’ll then utilize TensorFlow.js’s prediction functionalities to get prediction results for your recognized workouts, pick top scores, and return the recognized workout type. Finally, you’ll use Material UI and React.JS to display a complete workout history to the user.

skills learned: reusing code for PoseNet generated human pose data collection • running the inference • updating data stored in local storage

Cleaning and Exploring Text Data

In this liveProject, you’ll clean and analyze data scraped from Reddit to determine customer opinions of your products within a set time period. You’ll utilize common natural language processing techniques such as stemming, tokenization, and latent dirichlet allocation (LDA) to discover patterns in people’s opinions, and then visualize your results and summarize your findings.

skills learned: NLP preprocessing techniques such as stemming, n-gram, and removal of stop words using NLTK • visualize word cloud and top n-grams • perform topic modeling using LDA

Code the UI with Material

In this liveProject you’ll use ReactJS to build an entire user interface for a machine learning-powered exercise app. Your app records its users while they exercise, and so needs to have a short delay between them pressing a button and the start of recording. You’ll use Material-UI for React to implement layout, form elements, buttons, and user messages, and set up the UI logic for your delayed training data capture.

skills learned: adding Material-UI to React project • building grid layout structure • building page elements • building form elements

Collaborative Similarities

In this liveProject, you’ll build a movie recommendation system based on movies that your users have previously interacted with. This system is designed to boost engagement and keep your users on your site. You’ll develop a competitive array of similarity functions, and create your own recommender system based on these similarities. Finally, you’ll evaluate your system’s effectiveness and tune its parameters.

skills learned: read, process, and exploit user-item data • define and compute similarities between users and items using interactions • compute similarities between users and items in a recommender system

Conditionally Replace Text

As a PowerShell developer, your critical mission is to help XYZ Services company, the consulting and outsourcing firm you work for, avoid payroll delays after a massive failure in its IT infrastructure. To meet a requirement in the input process to payroll, the HR department needs an update to the state field in the source file to be changed from the employee’s state of residence to the state of the office they report to. Using PowerShell best practices, you’ll write a script that uses the switch statement, PowerShell cmdlets, and advanced regex expressions to read the input file, return a list of the unique states within the file, replace the state value according to the predefined conditions—and keep those paychecks coming.

skills learned: read text files • edit text using replace operations • regex patterns including lookaround • use of the switch statement to modify text • write to text files

Content-Based Similarities

In this liveProject, you’ll build a movie recommendation system based on the content and metadata of movies in your system. This system is intended to maximize the satisfaction of your movie-watching users. You’ll start with an analysis to determine the content of your movies, then use that data to implement content-based similarities for both products and users. You’ll build and evaluate your recommender system based on these connections, till it’s the best it can be!

skills learned: define and compute similarities between users and items using content and metadata • define recommender systems based on the definition of when two users or items are close

Cracking the Code

In this liveProject, you’ll help find a cure for a (fictional) global pandemic by identifying the origin of the virus. Biologists have already narrowed the list of suspects down to three animal species. Longest Common Subsequences (LCS) can help you measure the similarity between two sequences. You'll use them to compare the animal sequences with the virus’ sequence. You'll get a sense of what percentage of the sequences should be common versus what is actually common between the sequences studied, then you'll be able to identify the virus’ origin based on your findings.

skills learned: make simulations to compute “normal” LCS and identify outliers • determine how much is in common between each of the sequences from the outliers • draw conclusions from plots

Create a Color Blender

In this quick liveProject, you’ll build a custom graphic design tool that can generate a matching RGB color palette for a chosen color. You’ll create a blank canvas using NumPy arrays, display images using OpenCV, track a user’s activity with mouse events and flags, and also handle keyboard input.

skills learned: creating NumPy arrays • displaying images using OpenCV • using mouse events and flags to track user's activity on display window • handling keyboard input

Create a Data Platform for Real-Time Anomaly Detection

In this liveProject, you’ll close the gap between “data analyst” and “software engineer” by building a working data platform. You’ll join up with AnomalousDex Inc., a startup that specializes in personalized end-to-end data products, and create a working prototype of their anomaly detection platform to showcase to prospective customers. This requires connecting up multiple cross-discipline components, from data science to systems management.

This platform consists of three principal components: a service that serves the anomaly detection data model, the modeling platform, and a dashboard visualization tool. Your challenge is to develop all of these features, going hands-on with software architecture, data engineering, microservices, and dockerizing. You’ll also dive into the essentials of monitoring and metrics, and even train an unsupervised learning anomaly detection model!

skills learned: design a data platform architecture consisting of several Dockerized components • train an anomaly detection model with scikit-learn and deploy it in a web service • monitor the web service and anomaly detection model with Prometheus and Grafana

Create a Database Table

Logistica Transport has grown, which means they need to replace their outdated desktop app with a brand new web app solution. In this liveProject, you’ll help them kick off this transformation by swapping their current Excel-based storage system for a mySQL database. You’ll need to create and install the database from scratch, as well as set up the necessary tables to handle the input and output streams of data. This database is a vital foundation for how Logistica’s web app will function.

skills learned: MySQL language

Create a Neural Network

In this liveProject, you’ll use Keras to create a deep learning model for predicting basketball scores. Once your model is created, you’ll train it up on sample data and then validate your results to ensure it’s still accurate when applied to data from the real world.

skills learned: manipulate data with pandas • build a neural network with Keras • test and validate a neural network

Create a Product Index

E-commerce company Sneakers to the Max wants to strengthen its bottom line with an improved search engine that provides better and quicker results to customers. As a software engineer, your task is to create a structure for importing data into the search engine. You’ll write a Python script to import sneaker data into Elasticsearch. Next, you’ll create an index for the sneakers and use aliases to keep the data available while importing a fresh catalog. To verify your import and gauge the quality of your data, you’ll create a Kibana dashboard. Using Elasticsearch Bulk, you’ll improve the indexing performance of the sneaker data you need. When you’re finished, you’ll have created a complete structure for your imports that enables frequent, regular updates without downtime.

skills learned: import JSON objects from a file using standard Python language constructs • use Python built-in file IO mechanisms • interact with your Elasticsearch cluster using the Elasticsearch library for Python • verify if the import was run successfully using Kibana

Creating Features

In this liveProject, you’ll process raw data to make it ready for a machine learning model to diagnose diabetes rates. You’ll use feature engineering techniques to generate ML features from raw data. To make sense of your data, you will undertake data profiling, exploratory data analysis, analyze independent/dependent variables, and visualize data patterns. You’ll evaluate the correlation between dependent and independent variables to identify relevant features. You’ll even generate additional features as needed. Additionally, you will apply feature engineering techniques such as treating missing values and outliers to make your features ready for model training.

skills learned: use feature engineering techniques to generate ML features from raw data • data profiling, exploratory data analysis, analyze independent/dependent variables, and visualize data patterns • generate additional features

Cryptogram

In this liveProject, you’ll use Python to build a command-line game that produces a cryptogram to decode. You’ll build a simple web scraper to pull data from a quotation website, then generate an encoded puzzle for the player to decipher. You’ll interact with a web server to create your game, using the Python library BeautifulSoup.

skills learned: web scraping • error handling • basic encryption • modularizing code • requests • BeautifulSoup4 • debugging • exceptions

Customer Behavior

In this liveProject, you’ll clean and process data from Manning’s liveBook platform and prepare it for investigatory analysis to discover causes of customer churn. You’ll load a raw data dump of customer behavior data into PostgreSQL for analysis, categorize and check the event records for bad data, and calculate a basic set of interpretable customer metrics. Some SQL skills and a little Python is all you need to produce a clean and easy-to-interpret data set of customer behavior.

skills learned: loading data into a database • design and calculate customer metrics • perform quality assurance analytics

Customize AutoKeras Blocks

In this liveProject, you’ll build an AutoKeras block for image classification to help tune a neural network and customize the search space. You’ll then combine the block you have created with other prebuilt AutoKeras blocks to put together a complete automated machine learning pipeline.

skills learned: customizing an AutoKeras block for CIFAR-10 image classification • designing a hyperblock for model selection

DL for Text Classification

Predict the future! You’re an academic researcher working on a project that predicts what policy areas the U.S. government will prioritize. To achieve your goal, you’ll train three kinds of deep learning neural networks on a legislation dataset (a CSV file containing one row for every bill introduced in the U.S. Congress). With the resulting text classifications, you’ll predict the area of focus for future policy bills.

skills learned: sample datasets for unbiased measures of model performance • feature engineering for text data • fit models using Keras • evaluate classification models using appropriate metrics • tune hyperparameters to maximize model performance • explain how a machine learning model generated specific predictions

Data Augmentation for ML

As a machine learning engineer in an online recruiting tech company or HR department of a large organization, your task is to address a lack of data, a common problem in data science projects. To solve this, you’ll create multiple tools to augment processed data, increasing its volume and learning essentials about probability distributions, random sampling, and OOP. Completing this project will enhance your data analysis and visualization skills, taking you further down the path to a career in data science.

skills learned: data augmentation for ML • deal with missing data with statistical modeling • build data augmentation tools with OOP

Data Dashboards with R and Shiny

In this liveProject, you’ll develop a detailed dashboard for serving complex data to medical scientists working on a cure for cancer. Using the R language and Shiny framework, you’ll develop a dashboard that allows easy querying of different facets of the dataset using a web-based GUI. You’ll work to preprocess your dataset for efficient access, and refine and deploy the dashboard to your company’s web servers. When you’re done, you’ll have built a working prototype of an in-demand piece of data software.

skills learned: setting up a reproducible environment for an exploratory data project • creating an R dashboard package • deploying an R dashboard application

Data Ingestion and Cleaning

Imagine you’re a data engineer working at an enterprise. In this liveProject, you’ll set up a Databricks platform, creating clusters and notebooks, interacting with the Databricks File System (DBFS), and leveraging important Databricks features. You’ll also gain first-hand experience with Apache Spark—the world’s most widely used distributed processing framework—on tasks like reading the input data in CSV and JSON format, filtering, and writing the data to the data lake’s curated layer on DBFS.

skills learned: create clusters and notebook • interact with the Databricks File System (DBFS) • use Apache Spark to read input data in CSV and JSON format • understand the characteristics and importance of the first two layers of the widely used three-layer data lake design • use Databricks functionalities

Data Manipulation

Step into the role of a data engineer working at an enterprise. Your task is to build a data lake’s serving layer and ensure that business queries run on it in a matter of seconds. You’ll start with reading cleansed data that’s already sitting in the curated layer. Then you’ll transform it, enrich it, aggregate it, and denormalize it using Apache Spark. When you’re finished, you’ll have multiple output tables that make up the serving layer of the data lake.

skills learned: perform operations and transformations on data using multiple functions offered by Spark’s SQL module • use Spark’s UDFs to enrich data and process it in a user-defined way • compare the performance of different UDF-implementation approaches • understand the importance of the serving layer

Data Manipulation and ARIMA Modeling with Pyramid

In this liveProject, you’ll investigate seasonality in hotel cancellations by building an ARIMA model that can predict cancellations on a weekly basis. You’ll learn how to manipulate a dataset with pandas in order to form a weekly time series, before going on to make your first predictions.

skills learned: dataset manipulation with pandas • time series forecasting with pmdarima

Data Pipeline with Amazon Athena

Congratulations! You’ve just been hired as a data engineer for a mobile game development studio. The company’s modern data platform architecture includes an Amazon Athena data lake and an AWS Redshift data warehouse solution. Your task is to enable batch processing of revenue transaction data by creating an end-to-end data pipeline, connecting various data sources—including user engagement events, stage controls, and public chat messaging—to the lake house solution. Using AWS CloudFormation, you’ll provision the resources required for the data pipeline. You’ll connect a MySQL data source to the AWS S3 Data Lake and transform data in the data lake using Amazon Athena. You’ll wrap up the project by creating a dynamic analytics dashboard with AWS QuickSight. When you’re done, you’ll have built a batch-processing data pipeline, start to finish, using Amazon Athena.

skills learned: create a batch ETL pipeline from RDS (MySQL) to AWS S3 • extract and process data using AWS Lambda • run SQL queries programmatically in the cloud • create data environment configurations using YAML files • test AWS Lambda locally • create an optimized ICEBERG table in Athena

Data Preprocessing

In this liveProject, you’ll design a movie recommendation system step by step, from initial development all the way to a production-ready system. You’ll begin with preparing the data, then you’ll analyze the data, and finally, you’ll preprocess and export it. Along the way, you’ll gain a firm understanding of the data and your users, which is key for developing a system that makes appropriate recommendations. While this liveProject doesn’t contain any machine learning, going through these steps will prepare you for feature engineering and hyperparameter tuning later on.

skills learned: parse and engineer features using built-in and external Python libraries • use NLP to prepare the data and design features’ vocabularies for future embeddings

Data Preprocessing

In this liveProject, you’ll test your data wrangling and data processing skills using the Julia language. You’ll step into the role of a data scientist for a real estate company with a new task from your boss—analyze and clean housing and census data for the marketing and sales teams. You’ll employ the popular Julia package DataFrame.jl as well as powerful statistics related libraries to successfully explore these datasets, and prepare them for machine learning.

skills learned: tabular data ingestion and integrity validation • exploratory data analysis using descriptive and graphical techniques • feature selection and feature engineering

Data Processing Tools with NumPy

As a data engineer in an online recruiting tech company or a large organization’s HR department, you’ll build a series of practical tools to process and extract useful information from unstructured text data using NumPy. You’ll learn important methods (including trie data structure, TF-IDF, SVD), how to implement them, and their applications in the real world. When you’re finished, you’ll have the know-how to build data processing tools that meet the needs of machine learning engineers, data analysts, and product developers.

skills learned: extract features with NumPy • vectorize text with TF-IDF and SVD • compute similar items with embedded vectors

Data Streaming in AWS

As a data engineer for a mobile game development studio, your task is to create a data streaming pipeline that collects and processes large streams of data records in real-time for lightning-fast analytics. Your company’s modern data platform architecture includes an Amazon Athena data lake and an AWS Redshift data warehouse solution. To store files, you’ll create an AWS S3 bucket, and you’ll create an AWS Kinesis delivery stream by using the boto3 library to connect to AWS Kinesis endpoints and send event data to the service. You’ll provision AWS Redshift resources and connect them to your AWS Kinesis Data Stream to analyze user behavior data to understand the user's journey inside the app. When you’re done, you’ll have a simple yet reliable data streaming pipeline that prevents resource shortages and transforms data in real-time—while it’s still relevant—ensuring more accurate data.

skills learned: create streaming data pipelines on AWS using Kinesis and Redshift • provision the resources for your data pipeline using AWS Kinesis Data Stream • use AWS Lambda to simulate an event stream • create an AWS Redshift Serverless resource using AWS CloudFormation to transform data in AWS Redshift

Data Visualization for Exploratory Analysis

Visualize this: you’re a data analyst in an online recruiting tech company or a large organization’s HR department. You’ll use Matplotlib to explore, visualize, and analyze processed data to identify missing data and outliers. You’ll build interactive plots for superior data presentation, analyze the correlation of different features using visualization methods, and create analytics dashboards for two types of users. By the end, you’ll be a better data analyst and have the skills to build storytelling tools that let you answer important business questions.

skills learned: data visualization with Matplotlib • build interactive data dashboards • solve business problems with data insights

Decoding Data Science Job Postings to Improve Your Resume

In this liveProject, you’ll step into the life of a budding data scientist looking for their first job in the industry. There are thousands of potential roles being advertised online, but only a few that are a good match to your skill set. Your challenge is to use data science tools to automate the process of trawling through job listings, to save time as you optimize your resume, identify the most in-demand skills, and find jobs that are a good fit for you. To do this you’ll use Python to perform Natural Language Processing and text analysis on pre-scraped data from jobs posting websites.

skills learned: process data with pandas DataFrames • K-means clustering • information retrieval with tf-dif • text cluster visualization

Deep Learning

You’ve shown your expertise in successfully modeling a time series using classical models: moving average (simple MA and exponential MA), autoregressive (AR), and autoregressive integrated moving average (ARIMA). Your next step is to determine if there’s a way to boost the performance of the time series analysis. You decide you’ll apply an unconventional approach that’s increasingly popular in finance circles: a hybrid model that combines deep learning and classical approaches. For the deep learning models, you choose recurrent neural network (RNN) and long short-term memory (LSTM). In this liveProject, you’ll focus on preparing the data for the deep learning models. The good news is that in deep learning models, you avoid a long and cumbersome preprocessing stage since, unlike in many classical approaches, they are able to detect patterns almost automatically.

skills learned: prepare data for deep learning models • run RNN and LSTM deep learning models • visualize results • evaluate model performance

Deep Learning for Multimodal Image Retrieval

Have you ever spent too long scrolling through your photos to find that one picture you know you took? In this liveProject, you’ll solve that problem—with the power of deep learning!

Your challenge is to create a working image retrieval system that can search images based on text queries: type in ‘dog’ and you’ll get pictures of dogs. You’ll combine cutting-edge natural language processing and computer vision techniques, exploit semantic word representation methods, and train a neural network on social media data using image captions as annotations. This liveProject will develop your skills with PyTorch, and be an impressive piece for your data science portfolio.

skills learned: train a Word2vec word representation method • train a CNN with PyTorch • compute similarities between words and images

Deep Neural Networks for Image Segmentation in Fine Art

In this liveProject, you’ve taken on the challenge of digitizing the collection of the World Painting Museum, and the head curator wants you to use your machine learning skills to create an index of the art. Your challenges will include classifying your training data, considering pretrained models to help aid categorization, implementing a customized CNN that can identify genre, and applying image semantic segmentation in order to facilitate the classification by identifying the elements present on the art painting.

skills learned: image transformation • build an image classifier-based convolutional neural networks • apply transfer learning for image classification and segmentation • perform image semantic segmentation to increase accuracy of art paintings identification

Deep Reinforcement Learning for Self-Driving Robots

In this liveProject, you’ll investigate reinforcement learning approaches that will allow autonomous robotic carts to navigate a warehouse floor without any bumps and crashes. Your challenges will include teaching warehouse navigation with tabular Q-learning, leveraging neural networks and Deep Q-Network models to avoid collisions, and putting together a simulation environment to handle custom reinforcement learning problems.

Please note: This project recommends using Colab Pro, which is an additional cost outside this project.

skills learned: implementing Q-learning • improving algorithm performance with deep reinforcement learning • creating a reinforcement learning environment in OpenAI Gym

Defensive Distillation

Make your model less vulnerable to exploitation with defensive distillation, an adversarial training strategy that uses a teacher (larger, more efficient) model to learn the critical features of a student (smaller, less efficient) model, then use the teacher model to improve the accuracy of the student model. In this liveProject, you’ll use a pre-trained model to train your student model without distillation, generate malicious input using FGSM, and evaluate the undefended model. Then, you’ll train a teacher model with a pre-trained dataset, train your student model with the same training set and teacher model using distillation, generate malicious input, and evaluate the defended student model, comparing the results with and without distillation.

skills learned: basics of defensive distillation • basics of logit models • CleverHans attack generator

Demographic Data Analysis

In this liveProject, you’ll use the Python data ecosystem to explore how demographic data affects American life expectancy. You’ll use pandas and NumPy to clean and merge a newly collected data set Matplotlib, and seaborn to create a variety of plots that showcase the distribution of the data and expose the relationships between the variables.

skills learned: reading, cleaning, and merging data with pandas • exploring data for trends and distributions • doing data visualizations with Matplotlib and Seaborn

Deploy a DL Model

In this liveProject, you’ll deploy a completed deep learning model so that it can be easily used by your clients. You’ll use the Flask Python library to build a web page that will host your model, and use HTML, CSS, and JavaScript to build a simple web interface. Finally, you’ll take your model out into the world by using ngrok to make locally-served web pages available anywhere.

skills learned: create a web server with Flask • build a simple interface with HTML and CSS • make locally served web pages available to users on other systems

Deploy a Forecasting Model

In this liveProject, you’ll architect a solution to serve predictions from time series forecasting models over a REST API. Once you’ve architected your solution from a high-level perspective, you’ll monitor and assess the performance of the model and potentially undertake retraining to improve accuracy.

skills learned: serve our predictions with a REST API using FastAPI • validate the API query parameters by using Python’s type annotations and pydantic • track our model’s performance with MLflow

Deploy a PoseNet Model

In this liveProject, you’ll set up and tune a TensorFlow.js PoseNet model in a React web app. PoseNet allows real-time human pose estimation in your browser, all through the magic of TensorFlow.js. You’ll enable a WebGL backend, set up a system where PoseNet can utilize a webcam stream for its data input, and establish a canvas that can draw on top of an estimated pose.

skills learned: setting up TensorFlow.js PoseNet model in React Web app • enabling a WebGL backend • setting up Webcam live stream processing

Deploy a Predictive Model

In this liveProject, you’ll deploy a machine learning model to production so it can be easily used by colleagues. You’ll create a virtual environment for deploying your application, use Flask to build a local web service that returns predictions, and Heroku for remote deployment.

skills learned: create a virtual environment • GET and POST requests • create a web framework for a prediction app • remotely deploy an app on Heroku

Deploy a Predictor on Android

In this liveProject, you’ll create an Android application that can run a pretrained basketball predictor deep learning model for the easy use of your client. Your challenges will include converting the DataFrames into JavaScript arrays, converting your model into a TensorFlow Lite model, and finally packaging the model inside a working Android application.

skills learned: Converting a model to Android use • generating JavaScript usable data sets • simple Android development

Deploy a Predictor on the Web

In this liveProject, you’ll deploy a pretrained basketball predictor deep learning model onto the web for easy use by clients. You’ll utilize the powerful TensorFlow.js framework to ensure the model works in the browser, as well as converting DataFrames into JavaScript arrays, and building a simple website around the model and data sets.

skills learned: converting a model to web use • generating JavaScript usable data sets • simple web development

Deploy a Streamlit Dashboard

In this liveProject, you’ll build an interactive dashboard that will allow the marketing team at your company to monitor any mention of your company’s products on Reddit. You’ll start by visualizing relevant subreddit data, then build a model to monitor your mentions on Reddit. Finally, you’ll deploy the Streamlit dashboard on Heroku. Streamlit offers a simple and easy way to build a highly interactive and beautiful dashboard with just a few lines of codes, whilst Heroku offers free web hosting for data apps.

skills learned: dashboard design • data visualization • integrating API call with a dashboard

Deploying Models

In this liveProject, you’ll take on the role of an engineer at AISoft, where you'll be part of two dynamic teams: MLOps and Core-ML. On the MLOps team, you'll utilize software engineering techniques to ensure your models are not only accurate but also scalable, reliable, and maintainable. You’ve been assigned two important support tasks for the Core-ML team.First, you'll utilize the Gradio Python library to create an interactive web user interface that runs MRI classification and segmentation transformer models in the background, and then you'll build a pipeline that provides interpretability to the decisions of a construction vehicle classification model.

skills learned: build a web interface for MRI classification and segmentation using Gradio • run a vision transformer model for MRI classification and a Segformer or Maskformer model for MRI segmentation via the interface • implement Grad-Cam and LIME to explain transformer predictions

Deploying a Deep Learning Model on Web and Mobile Applications

In this liveProject, you’ll take on the role of a data engineer working for an app development company. Your boss has a great idea for a new app that can automatically identify what food is shown in an image—perfect for when you want to post your lunch on social media! They have asked you to build it for them.

Your challenge is to train an image classification system using TensorFlow and then deploy that system to production so that it can run in the web browser and as a mobile application. To do this, you’ll make use of the Python and TensorFlow ecosystems to create your model, then package it for deployment using Docker and Expo.

skills learned: train a deep learning model • deploy a deep learning model using TensorFlow.js • create web applications • deploy your application using Docker

Detect Eyes Using Haar Cascades and Hough Transformation

In this liveProject, you’ll upgrade facial recognition security to more complex iris recognition. You will read an image, draw bounding boxes around the detected eyes and detect the circular eye area region, then save the output image.

skills learned: reading and writing images • using Haar Cascade Classifier for eye detection • using Hough Circle Transform for circle detection

Detect Faces in Videos

Face detection has numerous applications across software and is an essential part of the pipeline for detecting deepfake videos. In this liveProject, you’ll implement a component that detects faces in videos, and normalizes them to the same size and visual appearance. This component is a vital foundation for a deepfake detection system.

skills learned: detect and crop faces • normalize and scale faces • align faces according to the eyes locations

Detect Sentiment with Transformers

Finative, the environmental, social, and governance (ESG) analytics company you work for, analyzes a high volume of data using advanced natural language processing (NLP) techniques to provide its clients with valuable insights about their sustainability. Your CEO has concerns that some of the companies Finative analyzes may be greenwashing: spreading disinformation about their sustainability in order to appear more environmentally conscious than they actually are.

As a data scientist for Finative, your task is to validate your sustainability reports by creating and analyzing them. You’ll compute conditional probability with Bayes’ Theorem, by hand, to better understand your model’s performance through metrics such as recall and precision. You’ll learn an efficient way to prepare your data from different sources and merge it into one dataset, which you’ll use to prepare tweets. To successfully classify the tweets, you’ll use a pre-trained large language model and fine-tune it using the Hugging Face ecosystem as well as hyperopt and Ray Tune. You’ll use TensorBoard and Weights & Biases to analyze and track your experiments, and you’ll analyze the tweets to determine whether enough negative sentiment exists to indicate that the company you analyzed has been greenwashing its data.

skills learned: conditional probability • fine-tuning a large language model • hyperparameter optimization • monitor training experiments

Detect a Face Using Haar Cascades

Welcome to the era of digital security! In this quick liveProject, you’ll develop an efficient solution for detecting faces in a live video stream. You’ll need to read an input video from a webcam, detect faces present in the input video stream, draw bounding boxes around the detected faces, and then write the detected faces to a new video.

skills learned: reading and writing videos • using a Haar Cascade Classifier for face detection

Detecting Unusual Data

In this liveProject, you’ll build your own Python data library that can detect unusual data. This data can affect regression analysis, and so your library will help spot it before it can negatively affect results. You’ll program functions to detect high-leverage points, outliers, and influential observations. Through the analysis of multiple examples, you'll gain a deep understanding of the kind of problems that can arise with real data.

skills learned: detecting unusual observations • visualizing and analyzing data with Matplotlib, seaborn, and statsmodels

Detection with SVM

In this liveProject, you’ll develop a machine learning solution that can identify the difference between faces in deepfake videos and real faces. You’ll use a support-vector machine (SVM) classifier approach to determine which videos have the artifacts associated with deepfakes, and combine face detection, feature extraction, and your SVM classifier together in one pipeline to create your detection system.

skills learned: Training an SVM classifier resulting in deepfake detection system • Evaluating a system on the test set

Develop Quantum Algorithms

As the researcher for venture capital firm Action at a Distance Corporation your job is to evaluate proposals from quantum startups hoping to secure an investment. But first, you need to expand your knowledge of quantum computing.

You’ll build your quantum muscles by investigating and coding key quantum algorithms including the quantum teleportation algorithm, the quantum phase estimation algorithm, the quantum Fourier transform algorithm, and Grover’s algorithm. With your new knowledge, you’ll compose a brief report about whether or not now is the right time to invest in quantum startups.

skills learned: Shor’s algorithm concepts/techniques • quantum superdense coding teleportation • quantum Fourier transforms • quantum phase estimation • quantum oracles

Develop Z-score and Baseline Results

Failure is not an option for Sigma Corp. As a lead data scientist for the large conglomerate of energy production companies, it’s up to you to help ensure interruption-free operations by developing a means for detecting anomalies that signal potential problems. Using metrics, including the receiver operating characteristic (ROC) curve and the area under curve (AUC) score, you’ll evaluate anomaly detection algorithms. You’ll build a z-score anomaly detection algorithm, which focuses on a single feature and provides a simple benchmark, and you’ll apply it to a dataset to establish a reference for comparison. When you’re finished, you’ll have a firm grasp of z-score anomaly detection, classification error categories, and evaluating anomaly detection algorithms.

skills learned: z-score method • evaluate algorithms using ROC AUC metrics • basic synthetic anomalies

Dictionaries

In this liveProject, you’ll utilize Python dictionaries to efficiently combine a file of product info and a file of sales entries. You’ll take on the role of a data engineer in a company that has been modernizing its sales systems, and tackle a common data merging task to help generate useful reports. Solve the challenges of this project, and you’ll understand the real power and flexibility of dictionaries and how they are used literally everywhere in Python.

skills learned: basic dictionary manipulation for adding and accessing data, in, get() • ways to handle missing keys

Difference in Differences

In this liveProject, you’ll use difference-in-differences as an alternative to A/B testing to assess the discounting strategy of an e-commerce company. This approach compares growth between a group that has been experimented on, and data from a control group. You’ll use Python and causal inference to examine the growth trajectory in profits from customers in a high discounts group against customers that didn’t get a discount.

skills learned: when and how to use difference-in-differences • estimate the impact of discount on profits using diff-in-diff

Dimensionality Reduction

In this liveProject, you’ll build a movie recommendation system that reduces the dimensionality of the spaces involved to enhance its performance so that latent representations are identified, which aim to capture users and items interactions with the system on the same latent vector space. You’ll work to compute clusters on the movies and clients of your site, and utilize latent representations of users and items with a reduced dimensionality to compute similarities. Once you’ve established this data, you’ll build and test your recommendation engine.

skills learned: compute clusters of users and items based on similarities • compute similarities on top of learned latent representations based on user and item interactions • define recommender systems based on similarity clusters

Dimensionality Reduction with PCA, t-SNE and UMAP

In this liveProject, you’ll use the Julia programming language and dimensionality reduction techniques to visualize housing sales data on a scatter plot. This visualization will allow the marketing team to identify links and demand patterns in sales, and is also a useful tool for noise reduction or variance analysis. You’ll use the popular PCA algorithm to visualize the sales dataset with overlaid clustering assignments from k-means and DBSCAN methods, and then expand Julia’s capabilities by calling Python modules using the PyCall.jl package. This extra flexibility will allow you to explore the t-SNE and UMAP algorithms which have excellent results for high-dimensional datasets.

skills learned: PCA, t-SNE, and UMAP dimensionality reduction techniques • validating and analyzing output of PCA algorithm • calling Python modules from Julia

Discovering Disease Outbreaks from News Headlines

In this liveProject, you’ll take on the role of a data scientist at the World Health Organization (WHO). The WHO is responsible for responding to international epidemics, a critical component of which involves monitoring global news headlines for signs of disease outbreaks. However, this daily deluge of news data is too huge to manually analyze. Your challenge is to pull geographic information from headlines, and determine where in the world outbreaks are occurring. Problems you will have to solve include extracting information from text using regular expressions, using the Basemap Matplotlib extension to visualize map locations for patterns indicating an epidemic, and reporting your findings to your superiors so resources can be dispatched.

skills learned: text extraction with pandas • clustering with K-means and DBSCAN • visualize clusters

Double Machine Learning Approach

In this liveProject, you’ll utilize machine learning for treatment effect estimation in order to help an e-commerce company deliver targeted discounting to the most profitable customers. You’ll build a model that predicts the effect of a discount on a customer following a causal model, maximizing profits for the business.

skills learned: pandas to manipulate tabular data • linear regression for causal inference • use machine learning to make causal models for personalization

ECOD Algorithm

Sigma Corp, a large conglomerate of energy production companies, has recently implemented anomaly detection algorithms and is generally pleased with their performance. However, analysts report that not all anomalies are being identified and the algorithms are too slow at times. As a lead data scientist at Sigma, it’s up to you to address these concerns. To increase the robustness of the algorithms, you’ll implement and optimize the probability-based Empirical Cumulative distribution-based Outlier Detection (ECOD) method, an alternative to statistical methods. You’ll benchmark the ECOD method in order to compare its performance with the statistical MD and PCA methods Sigma is currently using. When you’re finished, you’ll have firsthand experience implementing the highly efficient ECOD method to detect anomalies in multidimensional data.

skills learned: ECOD method

EDA for Text

Step into the shoes of an academic researcher tasked with predicting which areas will be the focus of the U.S. government’s policy-related efforts. In this liveProject, you’ll prepare for predictive modeling by exploring the policy areas and text descriptions in legislation data, using statistical visualizations and ggplot2, and identifying notable trends and outliers.

skills learned: import and clean data • generate basic data visualizations for time series datasets • tokenize and clean text data • calculate summary statistics for text data

Ensure Computational Governance

The development teams at Messflix, a movie and TV-show streaming platform are pushing domain data products through their newly implemented data mesh. Now the CTO has tasked you, their consultant, with striking a balance between the benefits of the data mesh, the freedoms the data products have, their usefulness, their interoperability, and aspects of their security. You’ll use Python and pandas to write policies that check the registration and pushed data, helping users provide all the required registration information. You’ll create tooling to classify data into categories for improved data labeling, and protect sensitive data with pseudonymization functions. When you’re done, you’ll have learned skills for federated computational governance that balance the benefits of data products with the benefits of the data mesh.

skills learned: understand computational governance • implement data policies for data products as well as individual data pieces

Escape Room

In this liveProject, you’ll use Python to build a simple text-based escape room game to help enhance the experience of a real-world escape game. This project involves following all the steps in a software development process, from reviewing your client’s given requirements, to designing, building, and testing your code. You’ll gain experience operating within a set of requirements and refactoring your code to optimize class definitions, tighten up functions, and make sure it’s Pythonic!

skills learned: software design • assertions • exceptions • error handling • refactoring • modularizing code • software quality • fulfilling software requirements

Estimating the Profitability of Discounts

In this liveProject, you’ll utilize causal inference techniques to help an e-commerce company estimate the impact of discounts on profits. You’ll learn how bias gets in our way of inferring the effect of discounts, and how to adjust for it for more accurate results. Leverage adjustment techniques, like linear regression, to adjust for bias and see how linear regression can improve the quality of your data. Your final task is to present a recommendation on whether the company should distribute its discounts or not.

skills learned: how bias gets in the way of causal inference • adjust for bias using a linear regression

Evaluate a Binary Classifier

Judging the effectiveness of a machine learning model requires in-depth analysis. This quick liveProject builds on the work you have completed in Machine Learning for Classification. You’ll assess your early models and consider better alternatives. You’ll plot the ROC curves of the model and compare it to multiple dummy models, and tune your hyperparameters to deliver the most accurate results possible.

skills learned: evaluate predictions and accuracy • visualize patterns with classes • conduct hyperparameter tuning • prepare a model for production deployment

Evaluate and Explain DL Models

In this liveProject, you’ll implement model performance metrics that can test the effectiveness of your models. You’ll calculate accuracy, precision, F1 score and recall values from the classification results for an existing model, and then estimate the ROC curve and AUC value. Finally, you’ll create a Gradient Class Activation Map. This map can highlight features and regions in an image that the deep learning model finds important, and manually inspect whether the model is performing in the desired way.

skills learned: make predictions using deep learning models using Keras • implement Grad-CAM visualization • implement deep learning model performance metrics • create a radar plot to display the performance of multiple models

Exceptions

In this liveProject, you’ll explore the unique way that exceptions work in Python. Python makes frequent use of exceptions, but they often function in a very different way than other programming languages. You’ll experiment with the basic try-and-except structure used for handling exceptions, then dive into using inheritance with exceptions, and creating your own hierarchy of exceptions.

skills learned: how Python exceptions are created, raised, and handled • what is in an exception object • how inheritance works for exceptions

Explore IMDb Dataset

In this liveProject, you’ll explore a dataset of movie reviews and prepare it for sentiment analysis. The dataset you've provided is balanced between positive and negative reviews, but is encoded in such a way that you will need to use the dictionary (lookup) in this dataset package to decode the content to plain text. Your challenges will include decoding the data to plain text, and then converting the plain text into tokens in a Pythonic manner.

skills learned: use a dictionary to decode and encode data • learn NumPy array indexing techniques • understand Python dictionary data structure, and how to edit and manipulate it

Extending ML for Text Classification

Imagine you’re an academic researcher working on a project for predicting trends in the U.S. government’s policy-making priorities. Using modern techniques for text data feature engineering, you’ll fit a set of models, subsample the training data to minimize bias, evaluate the models’ performance using a test-set of observations, and leverage a tidy workflow to explain how a model generates specific predictions.

skills learned: resample datasets • generate feature hashes for categorical variables • implement pre-trained word embeddings in a machine learning workflow • subsample an unbalanced dataset • evaluate classification models • explain how a machine learning model generates specific predictions

Extract Data from MySQL

In this liveProject, you’ll build an extraction pipeline to transfer data from a legacy server-side database in AWS MySQL into your new cloud platform. You’ll learn how to create an integration data layer, and develop a maintainable and flexible data intrastate that is both easy to work with and easy to scale.

skills learned: setting up a AWS Account • create a MySQL database instance • create an AWS Lambda function

Fake News Detection

In this liveProject, you’ll use the RoBERTa variation of the BERT Transformer to detect occurrences of fake news in a data set. Political news can be tricky to validate for accuracy, as sources report the same events from different biased angles. RoBERTa uses different pre-training methods than traditional BERT and has hyperparameters that are highly optimized, meaning it tends to perform better than its predecessor. You’ll start out by loading the model using the Hugging Face library and training it to your data with PyTorch Lightning. The project will also include the implementation of training a custom tokenizer from scratch and using it to tokenize the data. A successful model will maximize positives, and so model evaluation will be based on your model having a high recall score.

skills learned: loading and configuring pretrained RoBERTa model using Hugging Face • building and training a text classifier using PyTorch Lightning • validating the performance of the model and quantifying the results

Fast.ai

In this liveProject, you’ll create a recommendation engine for an online store using the Fast.ai library. You’ll utilize a dot product and a neural network to come up with the latent factors in a rating matrix, and compare and contrast them to determine which is likely to deliver the best recommendations. You’ll need to select and clean your data, pick the right methods, then create the functions that you need to recommend products based on predicted ratings.

skills learned: selecting, cleaning and choosing data for collaborative filtering • matrix collaborative filtering techniques using Fast.ai collab_learner class • learning latent factors for collaborative filtering recommenders

Feedback-Loop and Exploration Methods

Since recommender systems train and learn over the data they recommended themselves, they will never train over, learn, or recommend items that they didn’t already recommend for some reason, such as insufficient ranking to be seen by the user. It’s important to break this Feedback Loop in order to ensure that suitable recommendations aren’t missed. But you must strike a balance between deviating (just enough) from the system’s predictions through exploration and not defeating the system’s purpose altogether. In this liveProject, you’ll learn three methods of exploration that help you provide better recommendations to your users, as well as the costs and benefits of each.

skills learned: rerank a list of items after a model already predicted their rankings • add noise to the model to affect predictions as they’re made • combine lists with different rankings to increase traffic to lower ranking items • use a Feature Store as part of a prediction pipeline

Find Hidden Connections with Graphs

You’ve set your sights on a collection of prime off-the-market New York City properties whose value is likely to double in the next couple of years. With your excellent sales skills, you’re confident you can get the owners to sell at a fair price, if you can determine who the owners are. You have a knowledge graph—built from tax records, property deeds, and permits—that identifies entities associated with the properties. These entities might not be the true owners, but identifying them could help you determine who the true owners are.

Your task is to transform the knowledge graph into a representation that can be processed by a machine learning model later. Using simple graph and NLP techniques, including node2vec—one of the most influential algorithms in the graph community—you’ll improve the quality of the data by removing the noise from the knowledge graph. You’ll convert the nodes in your graph into embeddings, analyze how well your embeddings represent the underlying knowledge graph, and develop insights on tuning the node2vec hyperparameters. When you’re done, you’ll have learned techniques for analyzing and visualizing embeddings and associating them to the original graph, helping you determine who the “hidden” owners are.

skills learned: intermediate Neo4j (Graph Data Science library) • intermediate graph theory (weakly connected components) • build a node2vec model with Neo4j

Find, Replace, and Extract Text

XYZ Services Company, a consulting and outsourcing firm, recently experienced a massive failure in its IT infrastructure. While it’s working on getting the environment up and running again, the marketing director wants to email partners a report that includes contact information for all the marketing staff. As a PowerShell developer, your task is to write a PowerShell script that extracts properties from the plain-text employee directory file, uses those properties to build a custom PowerShell object, and exports the object to a CSV file which you’ll use to generate an HTML report that’s formatted for the marketing director’s purpose. As a favor to the HR department, you’ll also write a PowerShell script that corrects values in a specific field within the source file and writes the edited content to a new file.

skills learned: read text and import objects from a CSV file • write to a text file • basic regex • extract substrings from text • index arrays using expressions • create custom objects • export output to CSV and HTML formats • change PowerShell's default HTML output format

Forecast using Deep Learning

In this liveProject, you’ll use deep learning to implement powerful time series forecasting models that can beat the performances of previous models. You’ll work with the Python package “PyTorch Forecasting” and the deep learning models LSTM and N-BEATS. You’ll also get experience with key techniques of cross learning, ensembling, and hyperparameter tuning.

skills learned: perform forecasting with LSTM and N-BEATS models using PyTorch Forecasting and PyTorch Lightning • optimize models with Bayesian optimization using Optuna • implement callbacks for customized model training

From Data Lake to Lakehouse

Turn an existing data lake into a Lakehouse, using Delta Lake, an open table format (and the cornerstone of Databricks’ Lakehouse design). For data processing and interacting with the Lakehouse, you’ll use Apache Spark. As you transform the existing tables into Delta tables, you’ll explore Delta Lake’s rich features, see firsthand how it handles potential problems, and appreciate the sophistication of the Lakehouse design.

skills learned: leverage Delta Lake capabilities including schema enforcement and time travel • use Delta Lake commands • interact with multiple layers of the three-layer architecture • use Apache Spark for data processing and interacting with the Lakehouse

Functions as Objects

In this liveProject, you’ll create Python functions for specific data cleaning operations to help import product data from a legacy system into a new pipeline. You’ll work with a delimited text file with unhandy export conventions, and utilize functions to clean and convert fields for cleaner and seamless data transfer. You’ll even break your new operations down into separate functions so that you can easily reuse your code at a later date.

skills learned: functions as dictionary values • mutable objects as default function parameters • how to use and create decorators

GNN for Document Classification

In this liveProject, you’ll implement a Graph Neural Network (GNN). This powerful model will allow you to use the document content from the first liveProject combined with the structure of the citation graph from the second liveProject to build an even more powerful model—one that will predict the sub-field of statistics of each of your customers’ papers.

skills learned: node classification by text embedding signals and citation graph structure signals • build a GNN classifier with PyTorch Geometric

Generate Art

In this liveProject, you’ll replicate Google’s Deep Dream algorithm to explore the artistic creations of a neural network. You’ll start by investigating the latent space of a deep neural network and how manipulating its values can affect an input image. You’ll visualize inputs that maximize your filters, and manipulate these filters to produce ‘dream-like’ hallucinations.

skills learned: mathematical operations on images • implementing loss functions • transfer learning

Generate Text Samples

In this liveProject, you’ll build a deep learning model that can generate text in order to create synthetic training data. You’ll establish a data training set of positive movie reviews, and then create a model that can generate text based on the data. This approach is the basis of data augmentation.

skills learned: building and training a deep learning model for text classification • using sklearn module to report model classification metrics • condition based sampling technique for NumPy array

Generative Modeling with VAEs and GANs

In this liveProject, you’ll utilize unlabeled data and unsupervised machine learning techniques to build and train data generative models. You’ll employ generative modeling, such as a variational autoencoder (VAE) that can generate new images by sampling from the latent distribution. You’ll then use an unsupervised generative adversarial network to generate new images.

skills learned: implement autoencoders to reconstruct images • utilize variational autoencoders for image generation • train an unsupervised deep convolutional GAN

Get Started

Start building your robotics foundation: You’re a programmer helping create prototype robots for the RoboPrototypes company. You’ll get a brief introduction to ROS 2, set up the programming environment, and onboard robotics building blocks with basic tutorials. Then you’re ready to create launch files, which are integral to every ROS project. As you create packages in ROS 2, you’ll learn to define and use parameters and declare custom messages. When you’re finished, you’ll have a firm foundation for delving deeper into robotics.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: set up the programming environment for ROS 2 • ROS 2 fundamentals • create a ROS 2 workspace • create ROS 2 launchfiles, services, messages

Getting Started with Google Colab Using PyTorch

In this liveProject, you’ll get hands-on experience using the powerful Google Colab tool for machine learning and deep learning. Colab notebooks let you execute your data science code in Google’s cloud, getting all the benefits of Google’s incredible hardware. You’ll see how Colab works for yourself by running through simple machine learning tasks such as data preprocessing, making use of Colab’s free GPU and TPU hardware acceleration capabilities, and combining Colab with scikit-learn and PyTorch to train a classifier.

skills learned: using Colab as a Jupyter Notebook • utilizing Colab hardware acceleration capabilities • combining Colab with PyTorch

Getting Started with Google Colab Using TensorFlow

In this liveProject, you’ll get hands-on experience using the powerful Google Colab tool for machine learning and deep learning. Colab notebooks let you execute your data science code in Google’s cloud, getting all the benefits of Google’s incredible hardware. You’ll see how Colab works for yourself by running through simple machine learning tasks such as data preprocessing, making use of Colab’s free GPU and TPU hardware acceleration capabilities, and combining Colab with scikit-learn and TensorFlow to train a classifier.

skills learned: using Colab as a Jupyter Notebook • utilizing Colab hardware acceleration capabilities • combining Colab with TensorFlow

Getting Started with Jupyter Notebook

In this liveProject, you’ll get hands-on experience using Jupyter Notebook in a real-world data science project. You’ll train a simple KNN classifier and use Jupyter, IPython, and the easy-to-use Markdown markup language to document and share your work. Your challenges will include customizing your notebooks, incorporating your notebooks into a data science project, and sharing your projects with the community on GitHub.

skills learned: collaborate on Jupyter Notebook projects • customize your notebooks • work with Jupyter Notebook extensions

Graph Embeddings for Document Similarity

Now that one of the data scientists at your publishing company was able to extract citations from the full text of the representative text corpus, your task is to generate new vector embeddings based on the citation graph, then cluster the documents using these embeddings to gain insight from the graph structure.

skills learned: converting graph nodes to vectors • dimensionality-reduction • clustering

Graph Structure in Text Data

Imagine you work for a company that publishes scientific articles created by its customers—primarily researchers and scientists in the field of statistics—whose volume of research has grown so large that it’s not possible for your company to read every paper nor for the researchers to stay on top of everything happening in the field. Your task is to use automated means to find some structure in the collection of scientific articles so that the company can more easily home in on the researchers’ interests and help them research more effectively. To do that, you’ll generate document embeddings, use them to impute a document graph, and execute graph algorithms against this graph in order to generate insights from it.

skills learned: convert text into graphs • explore centrality algorithms for clustering using Neo4j

Graphs from Real-World Data

New York City real estate is a trillion-dollar market, and you’ve got your eye on a collection of prime NYC properties whose value will likely double in a couple of years. They’re not currently for sale, but with your excellent sales skills, you’re confident you can get the owners to sell at a fair price, if you can only determine who the owners are. To obtain the owners’ contact data, you’ll construct a knowledge graph from publicly available data that contains tax records, property deeds, and permits. You’ll scan the data for the target owners, analyze the datasets for possible relationships, develop a knowledge schema that can extract insights into your use case, and load the data into Neo4j to query and visualize the connections. When you’re done, you’ll have practical experience applying widely used graph tools to real-world data, and you’ll understand how different choices for your graph schema can lead to different insights.

skills learned: intermediate Neo4j (graph query using the Cypher language, loading and viewing graphs via Neo4j Desktop)

Handling Sensitive Data

In this liveProject, you’ll develop a machine learning model that can make a personalized prediction of whether an individual has COVID-19. You’ll work with a fictional audio dataset of digital footprints from smart phones, applying logistic regression and decision trees to turn this dataset into a COVID-19 predictor. You’ll learn how to formulate a ML problem by identifying data points, and their features and labels, so that you can take advantage of ready-made ML methods.

skills learned: basic flow control in Python • create and manipulate figures in Python • create and manipulate NumPy arrays to represent matrices and vectors

Hash Tables and Unit Tests

Your real estate developer client has a large sorted dataset and the ability to zoom in on select parts of the data, thanks to the programs you’ve written for them previously. Now they’ve noticed duplicate entries in the dataset, and they’ve asked you to solve this problem. In this liveProject, you’ll create a program that will read in the large CSV data file, leveraging a hash map (which you’ll test and implement) in order to discard duplicate entries as quickly as possible. Though you’ll use the csv and unittest libraries in Python, you’ll write your own unit tests—a fundamental software engineering skill and a practical requirement on most real-world projects. Finally you’ll test your tests against Python’s existing dictionaries.

skills learned: write code manipulating Python dictionaries • write unit tests involving dictionaries • cover a variety of testing scenarios • implement a basic hashmap from scratch • use your hashmap to eliminate duplicate entries

Hate Speech Detection

In this liveProject, you’ll use the ALBERT variation of the BERT Transformer to detect occurrences of hate speech in a data set. The ALBERT model uses fewer parameters than BERT, making it more suitable to the unstructured and slang-heavy text of social media. You’ll load this powerful pretrained model using the Hugging Face library and fine-tune it for your specific needs with PyTorch Lightning. As falsely tagging hate speech can be a big problem, the success of your model will involve calculating and optimizing its precision score. Your final product will run as a notebook on a GPU in the Google Colab environment.

skills learned: loading and configuring pretrained ALBERT model using HuggingFace • building and training a text classifier using PyTorch Lightning • validating the performance of the model and quantifying the results

How to Think about Automating Email

In this quick liveProject, you’ll automate the process of filling out a custom email from a CSV of email addresses and exam scores. You’ll then implement a randomizer to assign a random discussion partner, all using a Python library of your choice.

skills learned: using the smtplib library • reading documentation for Python • parsing a CSV file to store information in a data structure

How to Think about Manipulating Data

In this quick liveProject, you’ll use the pandas library to read a table from a Wikipedia page into a DataFrame, clean and arrange the data into columns, and extract useful information quickly to reveal data insights.

skills learned: organizing and visualizing data in a pandas DataFrame • reading, working, and experimenting with new Python libraries

How to Think about Scheduling Tasks

In this quick liveProject, you’ll develop a simple task scheduler app. Your challenges will include coding functionality that will arrange a day’s tasks to either minimize time left over during a day, or to provide the greatest variety of tasks during a day.

skills learned: implementing an optimization algorithm • recursion method to call a function • writing code to fit certain constraints • using brute force to find a solution

How to Think about Simulating Nature

In this quick liveProject, you’ll write a UI that simulates water drops on a window during rain. You’ll use nested lists, the random library, and tkinter to increase droplet size with certain probability, and resetting sizes when drops become too big.

skills learned: using the tkinter library • adding widgets to a UI • manipulating shapes in a UI • working with row and column data in a grid

How to Think about Solving a Maze

In this quick liveProject, you’ll design and implement an algorithm to solve a simple maze, revealing a path to the exit.

skills learned: implementing a wall-following algorithm • testing and debugging code • appropriate data structures • understanding algorithms, including edge cases

Human Pose Estimation with Deep Neural Networks

In this liveProject, you’ll take on the role of a machine learning engineer working for a company developing augmented reality apps. These apps include games, virtual shopping assistants, and fitness coaches that need to be able to reliably recognize the shape of a human body. Your challenge is to create an application for human pose estimation: detecting a human body in an image and estimating its key points such as knees and elbows. To do this, you’ll build a convolutional neural network from scratch, training your model using Google Colab and your GPU. At the end of this liveProject, you’ll have completed an interactive demo application that uses a simple webcam to detect and predict human keypoints.

skills learned: object detection and keypoint detection algorithms • transfer learning • training CNNs on images • deploy human pose estimator using webcam

Hybrid Model

Now that you’ve prepared the data for the deep learning models, you’re ready to apply the hybrid model, which leverages the strengths of both the classical and deep learning models. In this liveProject, you’ll focus on the essential steps for running a time series analysis. You’ll start with ensuring the data you prepared is ready for processing with the deep learning models. Next, you’ll run the RNN and LSTM for the separate datasets. Finally, you’ll evaluate the performance of your models.

skills learned: combine classical and deep learning models • visualize results • evaluate model performance

Identify Similar String Patterns

XYZ Services Company, the consulting and outsourcing firm you work for, recently experienced a massive failure in its IT Infrastructure. While it’s working on getting the environment back up and running, the company urgently needs to identify potential vulnerabilities in the digital signatures used by the employees for two-factor authentication. Applying PowerShell scripting best practices, you’ll write a script that uses the switch statement and its regex parameter to extract a list of digital signatures from an input file, identify matching strings that satisfy specific conditions, classify them as required, and send them to log files. When you’re done, the information security officer will thank you for helping prevent future security issues.

skills learned: read text files • use complex regex patterns including lookaround, regex quantifiers, and ranges • use the switch statement and its regex parameter to extract substrings from text files • arrays and array indexing using expressions • edit text using replace operations • write to text files

Image Classification

In this liveProject, you’ll join BrainAI’s MRI data analysis team. BrainAI needs you to develop a state-of-the-art AI module utilizing vision transformer models to detect brain tumors with exceptional accuracy. Armed with Python tools like Hugging Face Transformers, PyTorch, and more, you'll detect the presence or absence of tumors within human-brain MRI datasets. With Google Colab's GPU computing resources, you'll utilize deep learning to try and achieve a 95%+ accuracy rate.

skills learned: set up training, test, and validation datasets with biomedical (MRI) images • train and evaluate a vision transformer model for brain tumor classification

Image Classification with Deep Learning

In this liveProject, you’ll step into the role of a developer working for car rental company CarsUnlimited. You have a bold new idea for an app that can revolutionize your business. You envisage a service where a customer can snap a picture of any car on the street, and instantly find out if CarsUnlimited has that make and model available. Your colleagues have the app UI covered; your challenge is to build the algorithm to classify the cars that were snapped by the user. You decide to develop a machine learning algorithm for solving this problem.

You’ll start with cleaning your data, before experimenting with classical machine learning algorithms such as SVMs, Random Forests, Decision Trees, and assess their performance metrics. With an entire range of parameters to tune and improve, you will learn about how to automate this search. From there, you’ll dive into building a deep neural network and explore different search strategies to finalize your powerful new neural network architecture.

skills learned: identify when to use classical machine learning algorithms or deep learning for classification • build a neural network architecture and automate the search for parameters

Image Segmentation

In this liveProject, you'll pioneer the development of cutting-edge MRI segmentation algorithms using transformer architecture for computer vision company VisionSys. Manual segmentation is labor-intensive and expensive, so you’ll be developing a custom model that can do it for you. You'll train and evaluate SegFormer and MaskFormer models to identify brain tumor regions with over 90% accuracy. With Python tools like Hugging Face Transformers and Google Colab's GPU computing resources, you'll create pipelines, preprocess data, and showcase sample predictions and quantitative results.

skills learned: set up training, test, and validation datasets with biomedical (MRI) images • train and evaluate Segformer and Maskformer models for brain tumor segmentation

Implement Semantic Search with ML and BERT

In this liveProject, you’ll apply premade transfer learning models to improve the context understanding of your search. You’ll implement BERT (Bidirectional Encoder Representations from Transformers) to create a semantic search engine. BERT excels in many traditional NLP tasks, like search, summarization and question answering. It will allow your search engine to find documents with terms that are contextually related to what your user is searching for, rather than just exact word occurrence.

skills learned: create an inverted index with Faiss library • encode and search documents with a sentence-transformers library • complete question-answering tasks with pretrained BERT models

Import

In this liveProject, you’ll refactor a legacy module with unpredictable functionality to make it fit for your needs. The module behaves strangely whenever you import it, and so your task is to find the problem by exploring the ways that import in Python works. Your new knowledge will allow you to fix the issues in the test module, and create safer modules and multi-module packages.

skills learned: how the import process works • what importing only one feature of a module really does • how to make sure that code only executes when you want it to

Improve Search Support

As a software engineer at Sneakers to the Max, it’s your job to create an API that the e-commerce company’s developers can use to improve search results. You’ll write Python code to create Elasticsearch queries that work best for zero, one, or multiple search terms. To enable the application to interact with the search service, you’ll implement a FastAPI endpoint. You’ll add query filters, including colors and price ranges, to narrow down the results, and you’ll add pagination for when the number of sneakers in the narrowed search results is too large to be displayed on one page. When you’re done, you’ll have created an API that empowers your developers to provide the best matches to customers’ searches and leads to increased sneaker sales.

skills learned: use different queries to select sneakers

Inheritance

In this liveProject, you’ll explore how inheritance works in Python through developing classes that might be used in a simple video game. Your game follows animal characters moving through an obstacle course, with different movement styles for wolves, bears, eagles, ducks, frogs, and trout. In order to avoid having to repeat your code between characters, you’ll create some base classes and inherit from them. You’ll see how to inherit from more than one class at a time, and discover use cases for multiple inheritance.

skills learned: using Python’s system of inheritance • developing classes for a video game • using Python's multiple inheritance to create mix-ins

Initialize Qubits and Run Quantum Circuits

Enter the world of quantum computing and discover how quantum computers solve problems in a fraction of the time. You’re a researcher for Action at a Distance Corporation, a consortium of venture capitalists who believe the time to learn about and invest in quantum computing startups is now. Your task is to learn fundamental quantum computing skills so you can evaluate proposals from quantum startups and brief the consortium’s partners. Get familiar with the Qiskit environment as you create your own Qiskit programming Python imports template, then discover features of the Qiskit ecosystem as you add code to your template. You’ll set up a quantum circuit, getting first-hand experience with amplitudes and probabilities associated with quantum states. With these fundamentals under your belt, you’ll prepare brief notes for your report to the consortium, researching which companies will be the major players in the quantum race, what types of quantum hardware they’re pursuing, and what exactly is the “quantum advantage.”

skills learned: create quantum registers and circuits and begin to program them with Python and Qiskit

Intent Classification

In this liveProject you’ll develop a chatbot that can classify a user’s intent, using the Hugging Face NLP library. Your challenges will include building the task with the Bart transformer, and building a specialized task for detecting toxic language. You’ll then develop a Telegram bot and integrate it with your toxicity classification task.

skills learned: use a transformer to create a classification tool • build a chatbot using Hugging Face library to classify user messages • use transformers to create a classifier to identify toxic messages and integrate this classification into a chatbot

Interactive Superset Dashboard

Give your end-users what they want! In this liveProject, your challenge is to deliver an infrastructure that allows your end-users to query an existing, fully functioning modern Lakehouse for two different use cases: analyzing data aggregates over a long period of time to identify trends, and analyzing recently ingested data for monitoring purposes. You’ll use Preset, an SaaS platform that offers a managed version of Apache Superset, to run queries on the Lakehouse and build two interactive dashboards for your distinct use cases.

skills learned: use Apache Superset to interact with the data via its SQL editor • build charts, and add them to dashboards • create and configure a workspace on Preset and interact with its managed version of Apache Superset • translate business requirements and requests into SQL queries

Isolation Forests

Red alert! One of the energy production companies managed by Sigma Corp has suffered an outage. An investigation has led to the conclusion that the facility’s anomaly detection mechanism failed to detect early signals due to a sudden change in the distribution of the analyzed data. As a lead data scientist at Sigma, you’ll build an Isolation Forest algorithm, which is less likely than the Empirical Cumulative distribution-based Outlier Detection (ECOD) method to fail in such scenarios. To gauge how robust your method is, you’ll benchmark your algorithms against adversarial scenarios, synthetic anomalies, and standard datasets. When you’re done, you’ll have practical experience creating, using, and testing the Isolation Forest algorithm as an effective alternative to ECOD in circumstances where the data distribution changes.

skills learned: apply the Isolation Forest method • evaluate algorithms under adversarial distribution change

Iteration

In this liveProject, you’ll experiment with how iteration works in Python. You’re building a data analysis program that needs to iterate over a given number of Fibonacci numbers. To help out, you’ll construct an object that can be iterated over in a for loop or list comprehension that provides the Fibonacci series.

skills learned: build a data analysis program • use Python iteration protocol in creating iterables and an iterator using a generator function

K-means and DBSCAN Clustering

In this liveProject, you’ll use the Julia language and clustering algorithms to analyze sales data and determine groups of products with similar demand patterns. Clustering is a well-established unsupervised learning technique that’s commonly used to discover patterns and relations in data. You’ll apply k-means and DBSCAN clustering techniques to housing sales data for a retail startup, leveraging your basic Julia skills into mastery of this machine learning task.

skills learned: basic data preparation for clustering methods • clustering data with k-means and DBSCAN algorithms • evaluating and visualizing results

Latent Dirichlet Allocation

In this liveProject, you’ll use the latent dirichlet allocation (LDA) algorithm from the Gensim library to model topics from a magazine’s article back catalog. Thanks to your work on topic modeling, the new Policy and Ethics editor will be better equipped to strategically commission new articles for under-represented topics. You’ll build your text preprocessing pipeline, use topic coherence to find the number of topics, and visualize and curate the algorithm’s output for your stakeholders to easily read.

skills learned: implementing a simplified version of the LDA algorithm • preprocessing and converting a text corpus into a document-to-word matrix • generating topics using Gensim’s LDA algorithm • evaluating generated topics using Coherence and Diversity metrics • visualizing derived topics with a variety of techniques

Latent Semantic Analysis for NLP

At Finative, an ESG analytics company, you’re a data scientist who helps measure the sustainability of publicly traded companies by analyzing environmental, social, and governance (ESG) factors so Finative can report back to its clients. Recently, the CEO has decided that Finative should increase its own sustainability. You’ve been assigned the task of saving digital storage space by storing only relevant data. You’ll test different methods—including keyword retrieval with TD-IDF, computing cosine similarity, and latent semantic analysis—to find relevant keywords in documents and determine whether the documents should be discarded or saved for use in training your ML models.

skills learned: clean data with regular expressions • mathematical concepts and how and when to apply latent semantic analysis and cosine similarity

Learning Similarities

In this liveProject, you’ll determine what similarities there are between certain movies, and then build a recommendation system based on the similarities you’ve identified. Although you’ll be working with movie data for a streaming website, identifying similarities can help enhance recommendation engines for any item or product. You’ll develop a general technique for spotting similarities, then apply your learned similarities to build a recommendation engine. You’ll then enrich your model with collaborative and content-based data, and evaluate and compare your model.

skills learned: read, process, and exploit user-item data • define recommender systems based on similarities • analyze algorithm output

Leverage Embeddings

Create a powerful recommendation engine built from an ensemble of graph-based models that will help you tap into New York City’s real estate market by identifying groups of similar properties. You’ll start by working with transductive graph models (TransE and TransR) that are created specifically for knowledge graphs. Transductive learning takes observations from a specific set of training data and applies it to a specific set of test data. Next you’ll build an inductive model (GraphSAGE), which allows for generalized learning on new data (i.e. predictive modeling on previously unseen properties). Lastly, you’ll build the recommender system by using the k-Nearest Neighbor (kNN) algorithm to identify similar properties. When you’re done, you’ll have hands-on experience applying machine learning techniques to real-world knowledge graphs… and possibly a lucrative side hustle.

skills learned: build transductive graph models (TransE, TransR) with pyKEEN • build an inductive graph model (GraphSAGE) with Stellargraph

Life Expectancy Analysis

In this liveProject, you’ll perform a regression analysis on precleaned data to determine factors relating to American life expectancy. You will learn how to do a comprehensive regression analysis and how to select a model in the presence of multicollinearity. You will use the libraries developed in the second and third projects to check the validity of your final model.

skills learned: detecting multicollinearity with the Variance Inflation Factor • doing a comprehensive regression analysis with Statsmodels • visualizing relationships with seaborn • interpreting "QQ-plots," "Residuals vs. Fitted" plots, and "Scale-Location" plots

Link Front End and Data

Growth means change; for Logistica Transport, that means switching from their old desktop app to a new web app solution. In this liveProject, you’ll put together all of the pieces of this new web app. You’ll learn how to build a React.js app with Visual Studio 2023, set up a form for your users, and master the database magic of CRUD statements like insert, update, and delete to manipulate your database. Finally, you’ll validate your data formats to ensure the application works as expected.

skills learned: NuGet packages • Creating themed React forms • Implementing CRUD features • Implementing form validation

Lists

In this liveProject, you’ll utilize Python to create new data reports from a centralized system. You’ll work with raw data about the sales of various products, and massage it into a different format from which subreports can be extracted. To do this, you’ll make use of lists—one of the most common data structures in Python that are at the heart of data processing.

skills learned: list comprehensions • list slicing • ways to sort a list

Load Data into BigQuery

In this liveProject, you’ll use Google Cloud Platform’s BigQuery tool to transfer data from a legacy data lake to new cloud storage on GCP. You’ll start by setting up your Google Cloud developer account, then utilize Google Web Console to create a Cloud Storage bucket and load data into a BigQuery table. You’ll learn how to handle different file formats, and manage resources with Google Cloud Shell.

skills learned: setting up a Google Cloud Account • loading data into BigQuery • handling file formats

ML Pipeline with Amazon Personalize

Help your company’s messenger application provide better product recommendations for its customers. As a data engineer at the company, your task is to create a machine learning (ML) pipeline using the Amazon Personalize service. You’ll use CloudFormation templates to create a repository for the required AWS infrastructure resources, and AWS Glue to transform the raw user engagement data. Using Amazon Personalize, you’ll import a dataset and create and train the Amazon Personalize ML model for your users’ recommendations. To complete the project, you’ll create a workflow to train your Amazon Personalize recommendation solution using AWS Step Functions and user engagement events. When you’re done, you’ll have designed an ML pipeline using the Amazon Personalize API that provides product recommendations that suit your users best.

skills learned: create a data lake S3 bucket using an AWS CloudFormation template (Infrastructure as Code) and prepare data using AWS Glue • design a data pipeline to create and train an Amazon Personalize solution • deploy the pipeline using AWS Step Functions and AWS CloudFormation (Infrastructure as Code)

ML for Text Classification

Play the role of an academic researcher preparing a machine learning model to predict the U.S. government’s focus for new policy legislation. You’ll process the legislation dataset with resampling and feature engineering techniques, employ a range of algorithms, including penalized regression and XGBoost, to fit a series of ML models, evaluate the effectiveness of the models, and tune them accordingly.

skills learned: resample datasets for unbiased measures of model performance • feature engineering for text data • fit models using a tidy framework • evaluate classification models using appropriate metrics • tune machine learning models to maximize their effectiveness

Manage Data Products

As a consultant for Messflix Inc., a movie and TV-show streaming platform, you’ll investigate and discover why Meshflix’s recommender system breaks. You’ll brainstorm options for changes that don’t break the system, explore their pros and cons, and choose and implement one of your options. Then you’ll create an internal versioning strategy to support all the great product changes Messflix has planned for the future.

skills learned: understand product thinking • manage changes in data products • prevent breaks in downstream data products

Manipulate Data Distribution

In this liveProject, you will create an imbalanced dataset from the IMDb movie dataset. Your goal is to make a dataset where positive reviews are the minority. You’ll then test a theory that if you oversample positive reviews, you could rebalance the training data to build and train a text classification model. You finish up by examining the model performance with a confusion table, and basic metrics such as precision, accuracy and recall.

skills learned: building and training a deep learning model for text classification • using sklearn module to report model classification metrics • condition based sampling technique for NumPy array

Manipulate Qubits with Quantum Gates

Action at a Distance Corporation, a consortium of venture capitalists, believes that now is the time to invest in quantum computing startups. As its resident researcher, your job is to get familiar with quantum techniques and applications. You’ll code quantum circuits involving four different quantum gates, the fundamental processing units of quantum. You’ll visualize the effects of the gates, run and measure the circuits, and use IBM Quantum Composer to demonstrate the use of the gates. With a deeper understanding of quantum, you’ll investigate major quantum use cases to help the consortium determine whether or not the startups’ proposals have commercial applications.

skills learned: investigate and program quantum circuits and quantum gates • use IBM Quantum Composer to demonstrate the use of gates • program with Python and Qiskit

Measuring Bias in a Dataset

In this liveProject, you’ll compute and chart metrics that show the key characteristics in Home Mortgage Disclosure Act (HMDA) dataset, and investigate relationships between key demographics and other features that may lead to a biased machine learning model. You’ll compute and chart “equality of outcome” bias metrics, and produce a report into your insights.

skills learned: setting up a Google Colab environment • preprocessing a dataset using one-hot encoding • charting and visualizing metrics

Measuring Bias in a Model

In this liveProject, you’ll investigate and report on whether your company’s mortgage application classifier is making fair decisions between male and female applicants. You’ll train a logistic regression classifier on the HMDA dataset, compute performance metrics with a logistic regression classifier, and then chart “equality of opportunity” bias metrics.

skills learned: setting up a Google Colab environment • preprocessing a dataset using one-hot encoding • computing bias performance metrics for logistic regression

Methods for Multidimensional Datasets

Preventing operation failures and interruptions is mission-critical at Sigma Corp. The large conglomerate of energy production companies has recently implemented a z-score anomaly detection algorithm that focuses on a single feature. Now that the algorithm has proved its value, members of Sigma have requested additional algorithms that are just as simple to use, but that can handle multidimensional data. As a lead data scientist at Sigma, you’ll implement the Mahalanobis distance (MD) method and the principal component analysis (PCA) method as you build anomaly detection algorithms for multidimensional data. To gauge the performance of your algorithms, you’ll test them against a benchmark dataset as well as synthetic anomalies generated by your own algorithms. When you’re done, you’ll have firsthand experience building anomaly detection algorithms for multidimensional datasets as well as testing anomaly detection algorithms against both benchmark datasets and synthetic anomalies.

skills learned: PCA method • Mahalanobis method • evaluate algorithms using correlated synthetic anomalies

Mitigate Machine Learning Bias: A Health Data Project

Unexpected bias in machine learning models reduces accuracy, produces negative real-world consequences, and in the worst cases, entrenches existing inequalities for decades. Audits that can detect and mitigate this unwanted bias are an essential part of building and maintaining reliable machine learning systems.

In this liveProject, you’ll take on the role of a data scientist running a bias audit for the World Health Organization’s healthcare models. You’ll get to grips with state-of-the-art tools and best practices for bias detection and mitigation, including interpretability methods, the AIF-360 package, and options for imputing membership of a protected class. These tools and practices are placed in the context of a broad universe of bias where geopolitical awareness and deep understanding of the history and use of your particular data is key.

skills learned: biasing and debiasing ML models • using SHAP to interpret the underlying logic behind a ML model • using AIF360 to detect and mitigate unwanted bias • auditing techniques for unknown at-risk populations

Mitigating Bias with Postprocessing

In this liveProject, you’ll use the open source AI Fairness 360 toolkit from IBM and the “equalized odds post-processing” method to post-process your model for the purpose of bias mitigation. “Equalized odds post-processing” seeks to mitigate bias by making modifications to the prediction label after a model has been trained. After charting bias metrics on a basic classifier, you’ll tune the classification threshold to explore the impact on revealed biases.

skills learned: setting up a Google Colab environment • AIF360 open source library • post-processing for fairness with the “equalized odds'' method

Mitigating Bias with Preprocessing

In this liveProject, you’ll use the open-source AI Fairness 360 toolkit from IBM to measure and mitigate bias with model preprocessing. You will chart bias metrics on a basic classifier, before preprocessing your training data with the “reweighing” method. Reweighing seeks to mitigate bias by making modifications on the training data by computing and applying a set of weights. The weights are calculated such that the training data, with weights applied, is free of discrimination with respect to a protected attribute. Once your model is preprocessed, you’ll construct a classifier that makes use of the data.

skills learned: setting up a Google Colab environment • AIF360 open-source library • preprocessing for fairness with the reweighing method

Monitoring Changes in Surface Water Using Satellite Image Data

In this liveProject, you’ll fill the shoes of a data scientist at UNESCO (United Nations Educational, Scientific and Cultural Organization). Your job involves assessing long-term changes to freshwater deposits, one of humanity’s most important resources. Recently, two European Space Agency satellites have given you a massive amount of new data in the form of satellite imagery. Your task is to build a deep learning algorithm that can process this data and automatically detect water pixels in the imagery of a region. To accomplish this, you will design, implement, and evaluate a convolutional neural network model for image pixel classification, or image segmentation. Your challenges will include compiling your data, training your model, evaluating its performance, and providing a summary of your findings to your superiors. Throughout, you’ll use the Google Collaboratory (“Colab”) coding environment to access free GPU computer resources and speed up your training times.

skills learned: work with times series data • perform image segmentation • merge satellite imagery and perform operations on raster datasets • data augmentation for boosting model training • optimize and understand model performance

Named-Entity Recognition

In this liveProject you’ll develop a chatbot that can extract entities from user messages using the Hugging Face NLP library. Entity extraction pulls relevant data from chunks of text. Your challenges will include building a Named Entity Recognition task with the BERT transformer, and fine-tuning it to a medical context. You’ll then integrate your tasks into a chatbot.

skills learned: use a transformer to create a named-entity recognition tool • build a chatbot using Hugging Face library to extract entities from user messages like localizations and people

Navigation

Let your robot drive itself! Your company, RoboPrototypes, is preparing to demo its autonomous beverage delivery robot to investors soon, but the technician who was supposed to drive it has quit. Since there’s no one to take his place, your CEO has tasked you with upgrading the robot with automatic navigation. You’ll set up Nav2 to load your desired parameters, implement navigation planners and controllers, and optimize navigation by tuning costmap parameters. Once you send your waypoint goals to Nav2 telling your robot where to go, it’ll be ready to quench thirsts at a food court near you.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: planners • controllers • global and local costmaps • waypoint following • tuning Nav2 parameters

Nested Lists

In this liveProject, you’ll play the role of a programmer who has been hired by a real estate developer to help determine what a fair market price for rent should be across a number of owned properties. The good news is that the real estate developer has gathered data into a spreadsheet (in a CSV file). The bad news is that the spreadsheet is so large that they can’t even open it in a typical spreadsheet program, let alone do any sort of analysis with it. They’ve asked you to develop an interactive program that will perform some basic exploration of the data. You’ll use highly popular Python to tackle this challenge, and gain valuable experience processing CSV files and sorting lists as well as working with files, strings, and command-line arguments.

skills learned: develop a basic text-based spreadsheet program • read command-line arguments and perform file I/O • perform common string operations in Python • append items to Python lists • sort lists in Python using built-in routines

Neural Topic Models

In this liveProject, you’ll use the neural network-inspired Contextual Topic Model to identify and visualize all of the articles in a scientific magazine’s back catalog. This cutting-edge technique is made easy by the OCTIS (Optimizing and Comparing Topic Models is Simple!) library. Once you’ve established your text-processing pipeline, you’ll use coherence and diversity metrics to evaluate the output of your topic models, tune your neural network’s hyperparameters to improve results, and visualize your results for printing on posters and other media.

skills learned: exploring the vector representations produced by BERT • preprocessing text documents • using the Contextual Topic Model • performing hyperparameter tuning • evaluating topics using Coherence and Diversity and visualizing generated topics

Non-negative Matrix Factorization

In this liveProject you’ll use scikit-learn’s non-negative matrix factorization algorithm to perform topic modeling on a dataset of Twitter posts. You’ll step into the role of a data scientist tasked with summarizing Twitter discussions for the customer support team of an airline company and use this powerful algorithm to rapidly make sense of a large and complex text corpus. You’ll build a text preprocessing pipeline from scratch, visualize topic models, and finally compile a report of support topics for the customer services team.

skills learned: Implementing a simplified version of the NMF algorithm • preprocessing and converting text corpus into a document-to-word matrix • generating topics using scikit-learn’s NMF algorithm • evaluating generated topics using Coherence and Diversity metrics • visualizing derived topics with a variety of techniques

Nonogram

In this liveProject, you’ll use Python to build a nonogram puzzle generator that creates a printable puzzle pack from the command line. Nonograms are puzzles that reveal an image on a grid, based on numbers along the grid’s edge. Most programmatic implementations solve nonograms, but yours will generate them. To do this, you’ll code your script, relying on two essential data structures: the pandas DataFrame and the NumPy array. You’ll also experiment with image libraries including Pillow and pixelate, and gain a better understanding of basic image processing concepts.

skills learned: image manipulation • NumPy • table styling • file I/O • HTML • CSS • debugging • Pillow • pixelate • regular • expressions

Object Detection

In this liveProject, you'll spearhead the development of AI-aided surveillance software for construction site supervision. You’ll build two computer vision applications capable of detecting construction vehicles and their types across a large worksite and a more powerful model that can detect building defects such as cracks and fissures. Start by working with a pre-trained DETR model, then explore the Roboflow platform to assist you as you create a multi-class object detection dataset from multiple datasets with non-identical classes. With these datasets, you will train different transformer models for object detection to identify construction vehicles and cracks in buildings.

skills learned: import computer vision datasets from other platforms like Roboflow• merge two or more object detection datasets and their annotations on Roboflow • set up training, test, and validation datasets for object detection • train and evaluate three variants of DETR models for construction vehicle detection

Objects and Methods

In this liveProject, you’ll take on the task of extending a pricing application. You’ve just hit your first speed bump of needing to add functionality to the class that manages prices. However, that class is used elsewhere and you can’t get permission to change it. Your challenge is to implement a “monkey patch”—adding methods to the class at runtime—by experimenting with Python-bound methods.

skills learned: add functionality to a pricing application • create a class using type() • add bound methods to a class • monkey patch a class after creation

Optimize City Infrastructure

In this liveProject, you’ll use graph data optimization to determine improvements that could be made to the water infrastructure of Bruges. You’ll incorporate business data into your visualized Bruges street graph, and translate your graph into a prize collecting Steiner tree problem. You’ll then use the PCST solver in LynxKite to get an exact answer to your business problem.

skills learned: represent a network infrastructure problem as a graph optimization problem • the prize-collecting Steiner-tree problem • convert your business input data into the parameters of an abstract mathematical problem

Optimize Mapping

You’re a software engineer for Sneakers to the Max, an e-commerce company that sells sneakers. Its CEO wants to improve the speed and quality of the company’s search engine in hopes of boosting sales. Your task is to optimize the mappings of the Elasticsearch index. You’ll use the Kibana developer console to analyze the current mapping and create a new custom analyzer. To improve search matches—and customer experience—you’ll use the Elasticsearch field mappings to support multiple uses of the same field. You’ll modify the mappings to help customers find the sneakers they’re looking for, even if their search terms are incomplete or contain typos. To further improve CX, you’ll add synonyms to the Elasticsearch configuration to handle searches that use different terms for the same concept.

skills learned: mapping of different datatype fields in Elasticsearch • evaluating mapped datatype fields using analyzer tools

Pandas for Feature Extraction

Master the basic methods for handling most real-world scenarios as you play the role of a data scientist in an online recruiting tech company or a large organization’s HR department. Using pandas, you’ll process, extract, and transform numerical, categorical, time series, and text data into structured features that are ready for data analysis and ML model training. When you’re done, you’ll have hands-on experience working with most data types you’ll find in the real world, as well as useful skills for extracting and engineering features.

skills learned: feature engineering with pandas • cleaning, standardizing, and transforming data with pandas

Personalized Diagnosis of Symptoms

In this liveProject, you’ll apply the federated learning machine learning paradigm to compute a personalized infection risk diagnosis for COVID-19. Federated learning is an ML paradigm that learns from decentralized data via distributed computing environments. The risk diagnosis will come from a tailored classifier that is trained separately for each individual, by combining gradient descent with a network averaging method. You’ll then develop a federate learning algorithm that can be implemented by scalable passing messages over the contact network.

skills learned: use NetworkX to jointly represent network structured data and models • learn the weights of a linear classifier using gradient descent • implement a Federated Learning algorithm by combining gradient descent with a simple averaging scheme

Personalizing Discounts

In this liveProject, you’ll use causal inference to investigate data on randomized discounts and determine if an e-commerce company should offer personalized discounting. You’ll estimate a different treatment effect for each customer in the hopes to see if some are positive, and figure out which customers should get what discounts. You’ll utilize Python and machine learning to build this personalization system, and implement a causal model for personalization.

skills learned: implement causal models for personalization • evaluate the performance of your causal model using experimental data

Plotting with Seaborn

You’re a data analyst at Global Consensus Bureau, and your manager has asked you to determine how factors like education, hours worked per week, gender, and age relate to citizens’ incomes. Using pandas, you’ll pre-process data from a survey conducted by the U.S. Census Bureau. Then, you’ll create different charts to identify statistical relationships in data, using Python’s seaborn library, which provides custom functions for visual elements. When you’re done, you’ll know how to use seaborn to create visually appealing charts that reveal accurate and useful insights.

skills learned: visualize relations in categorical data • visualize relations in numerical data • visualize pairwise relationships

Predict Age Using GNNs

In this liveProject, you’ll build a model that can predict the age of customers for a telecom company. You’ll take basic profile data and call records for your customers, and build and improve a linear regression model using graph features. Finally, you’ll use a powerful graph neural network that can combine data from both profile features and graph structure to reliably reveal customer age range.

skills learned: using linear regression based on profile data • using linear regression with graph invariants as input features • using graph neural networks

Predicting Disease Outbreaks with Time Series Analysis

Time series analysis is an essential tool for data forecasting, allowing data analysts to make predictions about the future events and track relationships between data. In this liveProject, you’ll utilize the Python data ecosystem and time series analysis to analyze the spread of the COVID-19 virus in different parts of the globe.

Your goal is to make near-future predictions about virus spread based on your available data. You’ll start with an exploratory data analysis into the types of data you have access to, establishing the kind of questions it can reasonably answer. You’ll then develop an ARIMA model for time-series forecasting. Finally, you’ll develop an interactive Voilà dashboard using ipywidgets that will allow stakeholders to access and understand your analysis.

skills learned: time series forecasting with ARIMA • univariate and bivariate analysis • data visualization

Predicting Loan Defaults Using scikit-learn and H2O

The Python data science ecosystem is a powerful and open-source toolset utilized daily by thousands of data scientists and machine learning engineers. But with so many Python machine learning libraries to choose from, which tool works best for your needs?

In this liveProject, you’ll go hands-on with the scikit-learn and H2O frameworks, using them both to build working machine learning classifiers. You’ll use raw financial data and the tried-and-true random forest model to predict the chance of financial loan defaults. Once you’ve built your models, you'll compare implementations to find out which works best and evaluate your results against existing hard-coded tools.

skills learned: exploratory data analysis • working with pandas DataFrames • feature engineering • machine learning modeling with random forests • optimizing machine learning • model evaluation and comparison • deploying a model in a Python module

Predicting Urban Events with scikit-learn

A big city receives thousands of daily calls to its emergency services, reporting everything from illegal parking to life-threatening emergencies. In this liveProject, you’ll take on the role of a data scientist called in to advise on how the city can better allocate resources for these safety events. Your challenge is to create a machine learning model that can predict when and where different emergencies will occur. To do this, you’ll analyze data to identify trends, build and enhance a predictive model, and make your model explainable with model interpretability tools. You’ll also perform checks to ensure that the model won’t lead to bias or discrimination, and tweak your model so it can account for major lockdown events such as a pandemic.

skills learned: using machine learning to analyze and visualize location-based and time dependent data to make predictions • interactive data visualization • evaluating risks of bias and discrimination in production models

Prepare Tabular Data

In this liveProject, you’ll use the Python tools pandas and Folium to prepare the Airbnb dataset for training a deep learning model. You’ll learn Python concepts that ensure the application you build on this data is robust and maintainable, and implement geospatial and visualization techniques to illustrate the geographic distribution of Airbnb rentals.

skills learned: manipulate tabular data • create geospatial visualizations

Prepare Time Series Data

In this liveProject, you’ll create baseline models with Naive and sNaive methods for time series forecasts that you can use as a point of comparison for other models. Taking on the role of a data scientist for a large retail company, you’ll go hands-on to prepare sales data, create the baseline models, and optimize a Prophet model to compare against your baseline.

skills learned: build baseline models with Naive and sNaive methods • set up and optimize a Prophet model as an alternative model for comparison • evaluate various models using weighted MAE and MAPE

Preprocess Data in Parallel

The company you work for, which offers lifestyle products, has enjoyed success for several years, but management has decided it’s not competitive enough. Your task is to identify current market trends and new niche markets for the company’s lifestyle products. Using Python and Ray, you’ll build a web scraper that will load and save multiple web pages concurrently. To preprocess the data, you’ll read each of your locally stored pages, split them into sentences, tokenize each sentence with Hugging Face tokenizers, and store your tokenized documents in a new (pickled) format in your file system. When you’re finished, you’ll have leveraged Ray to preprocess a large amount of data while bypassing Python’s notorious concurrency limitations. The data science team will thank you for helping minimize the data preparation time, and the management team will thank you for helping the company sharpen its competitive edge.

skills learned: make your Python code concurrent or parallel • build a web scraper • pre-process texts for NLP models with Hugging Face transformers

Principal Component Analysis

Step into the role of data scientist at Finative, an analytics company that uses environmental, social, and governance (ESG) factors to measure companies’ sustainability, a brand new, eco-focused trend that's changing the way businesses think about investing. To provide its clients with the valuable insights they need in order to develop their investment strategies, Finative analyzes a high volume of data using advanced natural language processing (NLP) techniques.

Recently, your CEO has decided that Finative should increase its own sustainability. Your task is to develop a method to optimize the runtime for the company’s machine learning models. You’ll apply principal component analysis (PCA) to the data in order to speed up the ML models. To classify handwritten digits and prove your theory that PCA speeds up ML algorithms, you’ll implement logistic regression with scikit-learn. You’ll use the explained variance ratio to gain an understanding of the trade-offs between speed and accuracy. When you’re done, you’ll be able to present your CEO with proof of PCA’s efficiency in optimizing runtime.

skills learned: algorithm optimization with principal component analysis (PCA) • matrix manipulation with NumPy • nuances of scikit-learn's PCA library

Prophet Model Incorporated with Bayesian Analysis

In this liveProject, you’ll build a Prophet model that can forecast airline passenger numbers using data from the DataSF portal. The hotel you work for believes that analyzing the travel trends of US customers will help them forecast potential travel to Europe, and bookings in the hotel. You’ll enhance your model with "changepoints" that mark a significant change in trends, and make adjustments so your model can account for uncertainty in the trend and seasonality components.

skills learned: implementing prophet models • configuring changepoints • analyzing time series data with uncertainty interval configuration

Publish Data Products

Messflix Inc., a movie and TV-show streaming platform, wants to build a recommendation system for its movies and shows, but currently, its data landscape is too complex. As a consultant, your task is to implement a data mesh for an improved, accurate flow of its data. Using Python and JSON, you’ll help the data engineering teams sift more easily through Messflix’s data by creating separate, structured data products that can be pushed to the central data platform. From the organized data products, you’ll create a list of recommended movies, tailored to Messflix’s customers’ preferences.

skills learned: understand distributed domain ownership • produce data inside a data mesh • produce derived domain data products

Push Data into Data Products

Messflix Inc., a movie and TV-show streaming platform, is implementing a data mesh. So far, it has a self-serve data platform prototype where development teams can register their domain data products. As a consultant, your task is to build on that basic platform prototype with additional functionality: You’ll write a script in Python that will enable the development teams to push fresh data into their existing data products, write a function that adds support for versioned data, and implement a function that automatically calculates specific metadata (like row count and latest timestamp), then prints it to the screen. When you’re finished, you’ll have built a well-functioning, feature-rich, self-serve data platform, and be familiar with the requirements data-producing and data-consuming teams face daily and how to fulfill them.

skills learned: build a self-serve data platform • build a small data catalog • consume data inside a data mesh • store data products

Question Answering

In this liveProject you’ll develop a chatbot that can answer its user’s questions, using the Hugging Face NLP library. Your challenges will include building the task with DistilBERT transformer, and experimenting with other transformer models to improve your results. Once you’ve built an accurate NLP model, you’ll develop a Telegram bot and integrate it with your Question Answer task.

skills learned: use a transformer to create a question-answer tool • build a chatbot using Hugging Face library to answer user questions using Wikipedia data

Question Answering with Deep Learning

In this liveProject, you’ll step into the role of a data scientist working for an investment firm. Your company wants to make sure their investments meet European Union guidelines for environmental sustainability. That’s where you come in.

The EU taxonomy for sustainable finance is big, complex, and confusing. Your bosses need a program that saves them from searching through hundreds of pages whenever they have a query. You’ve been tasked with building a machine learning model that can pose certain questions to the EU guidelines, and return reliable answers.

Your challenges will include extracting text data from the EU taxonomy document, and matching environment questions with the corresponding paragraph in the guidelines. You’ll then set up a pretrained transformer Question-Answering model, evaluate its performance, and combine it with your question-paragraph model for an end-to-end solution. When you’re done, you’ll have an interface into which you can type a sustainable finance question and receive the correct answer from the EU guidelines.

skills learned: deep learning for NLP • information retrieval • extracting text from PDFs • working with transformers and auto-encoders • working with word and paragraph embeddings

Real-World Inference

Behind the scenes on websites like Amazon, Netflix, and Spotify, models make predictions on thousands of items every day. Then, based on what they’ve learned, they choose only the best recommendations to display for every individual user. In the real world, performing thousands of predictions one by one, as in a notebook-only model, would be highly inefficient. In this liveProject, you’ll reconfigure the models you implemented in the previous project to accept a list of items for each user and then evaluate all items at once—choosing the best recommendations much more quickly and efficiently.

skills learned: reconfigure a “notebook-only” recommendation system to a model capable of handling many-items per user • use linear-algebra to combine networks with differing dimensionality

Recognize Speech Commands with Deep Learning

Voice assistants and speech-based applications need a reliable understanding of human commands. Deep learning systems trained on audio data are a great way for models to learn directly from sound and easily classify what humans are telling them.

In this liveProject, you’ll become a data scientist prototyping a speech command system for a medical organization. Your company’s application will allow hospital patients to adjust their room TVs with their voice, and you’ve been tasked with building the core: a machine learning pipeline that can label audio inputs with given command words. You’ll tackle preprocessing and feature engineering methods for audio signals, expand your existing machine learning capabilities with audio data, and get hands-on experience with TensorFlow, Keras, and other powerful Python deep learning tools.

skills learned: performing digital audio processing • extracting spectral features from raw audio data • building deep learning architectures for audio

Regression Diagnostics

In this liveProject, you’ll build your own Python data library to check three of the assumptions of the regression model: normality, linearity, and constant variance. Confidence in your regression models depends on how well you have satisfied these assumptions. Once you’ve developed functions and plots that can check these assumptions, you’ll master techniques for correcting them. With this library, you will expand your data science toolbox with important diagnostics tools that will allow you to be confident in your results.

skills learned: checking the assumptions of the regression model • programming "QQ-plots," "Residuals vs. Fitted" plots, and "Scale-Location" plots • visualizing relationships and distributions with seaborn

Regression Discontinuity Design

In this liveProject, you’ll use regression discontinuity design (RDD) as a form of natural experiment that works as an alternative to A/B testing. RDD measures treatment effects at points of discontinuity to get an idea of the effectiveness of a program without needing to rigorously A/B test it. You’ll use this technique to assess the discounting strategy of an e-commerce company, in a way that will allow you to start applying causal inference to other practical problems.

skills learned: spot discontinuities in data • use RDD techniques to estimate the impact of discount on profits

Regression Using GLM and DecisionTree

In this liveProject, you’ll use the Julia language to build a regression-based machine learning model that can predict median house value in a neighborhood. You’ll start out with a simple linear regression model to give you a baseline value for quality metrics created with Julia’s package for Generalized Linear Models. You’ll then tune and assess a random forest model, and compare and contrast the two approaches to pick the best results.

skills learned: build linear regression and random forest models • evaluate models and explore their output • save model to the file for further reuse

Reinforcement Learning for Self-driving Vehicles

In this liveProject, you’ll develop an AI driving agent that can simulate independent driving. Your agent should be able to handle changing lanes, navigating roundabouts, and even self-parking. In order to create this complex agent, you’ll use powerful reinforcement learning techniques including model-based learning, model-free learning, and simulation-search/planning algorithms. Go hands-on to train your AI in AI Gym on Google Colab, improve your model’s performance, and provide professional-level documentation of your work for your colleagues.

skills learned: working with OpenAI Gym • developing reward-shaped training agents • using rl-agent and stable-baselines packages to directly train models

Report from Multiple Text Files

XYZ Services Company, the consulting and outsourcing firm you work for, is getting up and running after a massive IT infrastructure failure. In the meantime, the Human Resources department would like a report with basic staff data that it will use to distribute payslips to employees. As a PowerShell developer, you’ll use the switch statement and its file parameter to read and extract information from the source files provided by the company’s Active Directory before the system failed, build PowerShell objects, and export them into CSV format—helping HR provide proof of payment and putting your coworkers’ minds at ease.

skills learned: work with objects with FileInfo type • use the switch statement and its File parameter to process multiple text files • create custom objects • add properties to objects • use methods to modify text strings • export objects to CSV format

Retrieval and Ranking

Real-world recommendation systems work with millions of users and billions of items. To handle this massive scale, recommendation systems tend to be divided into two types of models (retrieval and ranking), running one after the other, narrowing down the set of items each time. In this liveProject, you’ll implement a retrieval model using TensorFlow Recommenders, combine it with a ranking model, then create a fully functional pipeline going through both models, using a Feature Store. Finally, you’ll explore the scenario where you can’t (or choose not to) run both retrieval and ranking models online in real-time, leveraging the Feature Store once more to use the retrieval model offline.

skills learned: split a single recommendation system to two essential parts, retrieval and ranking, using TensorFlow Recommenders • understand what Feature Stores are and how to use them

Run Circuits on Quantum Hardware

Action at a Distance Corporation, a consortium of venture capitalists, believes that the time to invest in quantum computing startups is now. As the consortium’s researcher, your task is to gain knowledge of quantum computing capabilities.

You’ll explore matrix representations—mathematical underpinnings of quantum computing—and examine quantum circuits that use different arrangements of gates to achieve identical functionality. You’ll upgrade your quantum toolbox with phase kickback, an essential component of many quantum algorithms, and you’ll work with Bloch spheres, a way of visually representing qubits. You’ll then use two different methods to run a circuit on IBM quantum hardware, a task that will be commonplace in the not-so-distant future.

skills learned: quantum gates as matrices • circuit identities • phase kickback • Bloch spheres • run programs on IBM quantum hardware systems on quantum computers

Run Two-Qubit Circuits

Get tangled up in quantum computing! Action at a Distance Corporation, a consortium of venture capitalists, believes the time to invest in quantum computing startups is now. As the consortium’s resident researcher, your task is to understand the power of quantum computing.

You’ll learn to recognize normalized quantum states of two qubits, a key skill in implementing quantum algorithms. You’ll get familiar with the Controlled Not (CNOT) gate, vital to the construction of important, entangled states, and you’ll observe what Einstein called “spooky action at a distance” as you program the (famous!) entangled Bell states. Using the IBM Quantum Composer, you’ll visualize and animate quantum circuits, gaining insight into handling gates and their outcomes.

skills learned: programming circuits involving 2 qubits • quantum gates • Bell states • IBM Quantum Composer

Run a Real-Time ML Model

EKKo Inc., the machine learning consultancy you work for, has been building an embedded system to enable deaf or hard of hearing people to participate in online meetings and events on their mobile devices. As a data scientist, your task is to complete the system by enabling it to detect American Sign Language (ASL) in real time. You’ll define a CNN model, train it using the existing ASL dataset, and evaluate the model’s accuracy. To optimize training speed and quality, you’ll fine-tune the model’s hyperparameters using the Keras Tuner. You’ll enable the model to be quantization-aware with the TensorFlow Model Optimization Toolkit (MoT), optimizing size and CPU consumption while maintaining model accuracy. To complete the project, you’ll connect the quantized model with a live video stream and train it on the embedded system. When you’re done, you’ll have a fully functional quantized CNN you can run on an embedded system that successfully detects and transcribes ASL in real time.

skills learned: set up a real-time training algorithm on an embedded board • train, evaluate, and tune hyperparameters on an embedded board • workflow of model tuning from the desktop to the embedded board

Search Algorithms

The real estate developer you’ve worked with previously is so pleased with your work that they’ve asked you to write a new program that allows them to zero in on specific data in sorted reports. Their ultimate goal is to determine what kinds of homes fall into different ranges of interest. Using Python, the csv library, and a custom ADT (that you’ll design), you’ll implement multiple distinct search algorithms as you create a program that allows your client to run many queries— and be fast and efficient, despite the large size of the dataset.

skills learned: design an ADT that abstracts over a dataset and implement it for sorted and unsorted Python lists as well as balanced and unbalanced binary search trees • implement a routine to search through the data • check the correctness of your code using provided unit tests

Semi-Supervised GANs for Melanoma Detection

In this liveProject, you’ll utilize PyTorch and powerful semi-supervised learning techniques to construct an advanced image classifier that can tell whether a 32x32 pixel photo of a mole is melanoma-positive—despite working with a very small labelled dataset. You’ll set up your image preprocessing pipeline, feed data into your PyTorch model, and then train a semi-supervised GAN model on both labeled and unlabeled datasets.

skills learned: generative modeling with unsupervised GANs • semi-supervised learning with GANs

Sensors and Sensor Fusion

You’re a programmer at the RoboPrototypes company, and its “dribot”—a prototype of a robot that will serve drinks to customers at a food court—has been running into things (including the CEO’s shin). Using Gazebo 11 and RVIZ2, you’ll add LiDAR (light detection and ranging) and IMU (Inertial Measurement Units) sensors that will provide a view of the environment from the robot’s perspective. To make the most out of the sensor data, you’ll fuse information from multiple sensor sources to better estimate where the robot is in the world.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: basics of sensors (encoders, IMUs, LiDARs, ultrasonic sensors) • URDF/XACRO • create a robot description • simulate sensors • sensor fusion

Sentiment Analysis and Natural Language Processing for Marketing

In this liveProject, you’ll step into the role of a Natural Language Processing Specialist working in the Growth Hacking Team of a new video game startup. Your team wants to massively accelerate your company’s early growth by acquiring huge numbers of customers at the lowest possible cost. To help tailor marketing messages, your boss has asked you to map the market and find out how customers evaluate your competitors’ products. Your challenge is to create a sentiment analyzer that will give a deeper understanding of customer feedback and opinions. To do this, you’ll need to download and create a dataset from Amazon reviews, build an algorithm that will determine whether a review is positive or negative, evaluate your analyzer's performance against star ratings, and lay out your findings in a report for your manager.

skills learned: perform sampling from imbalanced data • dictionary-based sentiment analysis • analyze reviews with deep learning • compare classifier performance

Serve a Compound Model

The company you work for, which provides a news feed aggregator, is plagued with an influx of hoaxes that are putting its reputation in jeopardy. The data science team has already trained a set of complex natural language processing (NLP) models to distinguish real news from fake news. Your task is to build a service, using Ray, that exposes the endpoint that returns the JSON object categorized as either a hoax or news. Then, you’ll optimize the service for performance and speed, enabling it to perform more parallel operations and use as many GPUs as possible. When you’re finished, you’ll have firsthand experience using some of Ray Serve’s advanced features for serving and optimizing a compound model—and you’ll have kept your company’s reputation safe.

skills learned: use Ray Serve • use FastAPI • NLP text preprocessing • use Hugging Face transformers to transform text

Serve a Model

Step into the role of a machine learning engineer working for a healthcare company that provides software to hospitals. One of your clients, a national health provider, has asked your company to provide software that predicts heart failure in patients. Using scikit-learn, you’ll develop a model that uses linear regression on a public Kaggle dataset containing heart failure data. Using Ray Serve, you’ll first deploy a function that accepts a wide range of parameters, then serve your model and provide functionality for multiple concurrent requests. When you’re done, you’ll have learned to use the Ray framework to serve your model through a webpage and helped your client save lives by using its patients’ parameters to predict imminent heart failure.

skills learned: use Ray Serve • use FastAPI integration with Ray • deploy a scikit model with Ray Serve

Simple Shop Stock Tracker

Imagine that you’re a programmer hired by a shop to build a basic digital infrastructure. In this liveProject, you’ll use Julia to create a simple stock tracker that implements the four steps of the shopping process. You’ll start by building a structure for your shop, then you’ll write a function to manage inventory delivery as well as one to keep track of items sold. You’ll wrap up by creating a function to handle cash transactions. When you’re done, you’ll have hands-on experience using basic Julia tools including functions, arrays, and dictionaries, as well as more advanced tools such as multiple dispatch and composite types.

skills learned: basics of Julia • functions • dictionaries • structs • properties

Simultaneous Localization and Mapping

Keep tabs on your robot. RoboPrototype, the company you work for, is preparing to demo its autonomous beverage delivery robot for investors in a food court next month. Your CEO has tasked you with setting up SLAM (Simultaneous Localization and Mapping) on the robot, making it possible to locate the robot at all times. Using the Slam Toolbox, you’ll equip the robot with the ability to create a map of its environment as well as pinpoint its location on the map. You’ll also get firsthand experience mapping on real data captured on a Turtlebot3 Burger, a widely used mobile robot platform.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: map with slam_toolbox • localization with slam_toolbox • SLAM in the bag

Slap

In this liveProject, you’ll build a two-player version of the card game Slap that can be played from the command line. To code this game, you’ll use algorithms, object-oriented programming concepts, and event listeners. You’ll gain hands-on experience importing modules, creating classes, defining methods, and using other principles of object-oriented programming.

skills learned: algorithmic design • software design • event listening • error handling • assertions • exceptions • modularizing code • debugging • pynput

Sorting Algorithms

The real estate developer you’re working with wants a program that can non-interactively read, sort, and write out the sorted data to a new file. In this liveProject, you’ll create this new program using the csv, abc, and time libraries. But instead of relying on Python’s built-in lists, you’ll implement the program with your own custom lists and sorting routines—gaining hands-on experience working with commonly used data structures and algorithms, demystifying all the work Python does for us and impressing your client, all at once!

skills learned: define an abstract data type (ADT) • define a linked list • implement sorting algorithms • perform algorithmic time complexity analysis

Spam SMS Detection

In this liveProject, you’ll use the DistilBERT variation of the BERT Transformer to detect and block occurrences of spam emails in a data set. You’ll utilize binary classification to determine whether an email is spam, or legitimate. The DistilBERT model uses knowledge distillation to highly reduce the size of the transformer model, thus optimizing time and resources. You’ll learn to use the HuggingFace library to load your data set, and fine-tune it to your task with PyTorch Lightning. You’ll also explore alternative training approaches that utilize novel APIs in the transformers library to fine-tune pre trained DistilBERT models. Every part of an NLP pipeline is covered, from preprocessing your data to remove symbols and numbers, to model training and validation using F1-scoring to assess the robustness of your pipeline.

skills learned: loading and configuring pretrained DistilBERT model using Hugging Face • building and training a text classifier using PyTorch Lightning • validating the performance of the model and quantifying the results

Summarization

In this liveProject you’ll develop a chatbot that can summarize a longer text, using the HuggingFace NLP library. Your challenges will include building the task with the Bart transformer, and experimenting with other transformer models to improve your results. Once you’ve built an accurate NLP model, you’ll explore other community models and integrate your summarization task with a chatbot.

skills learned: use a transformer to create a summarization tool • experiment with different summarizers models • build a chatbot using the Hugging Face library to summarize Wikipedia articles

Summarize COVID-19 News Using NLP and PyTorch

Text summarization is a powerful data science technique. It allows vital high-level information to be automatically extracted from reams of text data, without any slow and expensive human analysis.

In this liveProject, you’ll master text summarization techniques for summarizing news data by building a tool that can extract key information about COVID-19 from news articles using the PyTorch deep learning library. Your challenges will include cleaning your data set, building an attention-based deep learning model, and evaluating the success of your model using TensorBoard and ROUGE scores.

skills learned: deep learning for NLP • model visualization with TensorBoard • sequence-to-sequence modeling • text summarization with attention techniques

Summarize News Articles with NLP and TensorFlow

News Media Corp needs to be quick if they want to get ahead of their competitors. Their current news frontpage is put together manually, in a time consuming process where human editors create flashcards that summarize articles. It’s too slow—so senior management wants to supercharge the process using natural language processing. To get this built, they’ve turned to you. Your challenge in this liveProject is to create an NLP model that can reduce turnaround time for news editors with an automatic text summarizer. To do this, you’ll need to prepare and process your dataset with tokenization and padding, extract meaningful statistics from it, and finally use your dataset to train a deep learning model that can speedily summarize a body text.

skills learned: convert an abstractive text summarization dataset to an extractive one • train a deep learning model to perform extractive text summarization

Surprise

In this liveProject, you’ll create a product recommendation engine for an online store using collaborative filtering techniques from the Surprise library. You’ll work with Amazon review datasets to create your data corpus, and identify which would be best for a collaborative filtering recommender. You’ll then use two different approaches—neighbourhood-based and matrix factorization—to implement different solutions to the rating matrix completion problem. You’ll learn how to select and clean the necessary data for these different approaches. When you’re finished, you’ll have built a system that can predict the rating for a product a user has not yet purchased.

skills learned: selecting, cleaning and choosing data for collaborative filtering • neighborhood and model based collaborative filtering techniques with the Surprise library

Targeted Attacks on Your Classifier

Mount a targeted attack! Your goal is to mislead an existing DL model into predicting a specific incorrect target class. First, you’ll load your dataset, learn its structure, and examine a few random samples using OpenCV or Matplotlib. Next, you’ll prepare your dataset for training using NumPy. Then you’ll generate malicious input using three different classes from the highly popular CleverHans attack library. Finally, you’ll enlist NumPy again to evaluate the success ratio of your attacks.

skills learned: basics of targeted adversarial attacks (PGD, BIM and Carlini & Wagner) • Clever Hans attack generator

Test Functionality

Logistica Transport is on the rise—and they need a new web app solution that can do what their old desktop app can’t. That app is almost ready, but it needs a final round of testing to make sure it’s ready to go to its users. That’s where you come in! You’ll learn to utilize mocks and stubs, as well as the Jest and React Testing Library, to spot errors and make sure the app is production-ready. You’ll even test and assess the app’s performance using BenchmarkdotNET.

skills learned: Visual Studio debugging tool • Testing applications • Checking application performance • Entering application accessibility features

Text Search with spaCy and scikit-learn

In this liveProject, you’ll explore and assess essential methods for unstructured text search in order to identify which is the best for building a search engine. You’ll preprocess the data for this task using the spaCy library, and then experiment with implementing both a TF-IDF search and an inverted index search to find relevant information.

skills learned: apply Python’s spaCy library to perform essential natural language processing steps • compute TF-IDF tables and apply term frequency search to them • build an inverted index

Time Series Components

Imagine you’re a consultant with a number of prestigious financial institutions on your client list. One of them has hired you to increase the accuracy of their model for predicting stock prices. You rise to the challenge, deciding on some classical time series models. But before you can propose a reliable model, you must decompose and examine the time series data in order to understand its pattern. Once you have a firm grasp on the data’s peculiarities, you’ll be ready to run the time series modeling.

skills learned: use Alpha Vantage API • explore data using ACF, PACF, and ADF tests • deal with non-stationary data

Time Series Forecasting in Python

Forecasting is one of the most useful techniques a data scientist can bring to an organization. By making predictions from complex data, they can guide policy and resource management and allow businesses to plan and prepare for the future.

In this liveProject, you’ll take on the role of a data scientist who’s been tasked with forecasting the future consumption of an energy company’s customers. Energy consumption is seasonal and complex, and so you’ll be using the powerful Python ecosystem to tame your data and make your predictions. Your challenge is to use regression and time series techniques to forecast and visualize hourly energy usage for millions of customers, and present your findings in a clear report to allow your bosses to oversee resource allocation.

skills learned: visualizing complex relationships between variables and across time • build linear regression and time series models (exponential smoothing, ARIMA) with statsmodels • adding intervention terms to time series models

Time Series Modeling with TensorFlow Probability

In this liveProject, you’ll combine the power of deep learning with probabilistic modeling. You’ll build a structural time series model that can develop probabilistic forecasts of hotel cancellations, and use this model to identify anomalies across your cancellation data. You’ll perform a similar analysis of an air passenger dataset, and then use Bayesian Switchpoint analysis to determine the approximate time interval in which searches for the term “vacation” declined.

skills learned: structural time series modeling • forecasting and detecting anomalies • conduct Bayesian Switchpoint Analysis

Traditional Analysis

Now that you’ve detected the time series components and obtained the stationary data, you’re ready to move on to time series modeling. In this liveProject, your challenge is to determine which model will perform best for your client’s data. To do this, you’ll apply the classical moving average (simple MA and exponential MA), autoregressive (AR), and autoregressive integrated moving average (ARIMA) models. Then you’ll compare their performance using a visualization and performance metric, root mean square error (RMSE).

skills learned: model time series using moving average (simple MA and exponential MA), autoregressive (AR), and autoregressive moving average (ARIMA) models • visualize and compare performance using root mean square error (RMSE)

Traffic Sign Classifier

Tackle a fundamental step in many AI applications: building a simple image classification model. Using Convolutional Neural Network (CNN) layers, you’ll create this deep learning model for victims of adversarial machine learning attacks, train it on a publicly accessible traffic sign dataset, and implement it using Python.

skills learned: basics of OpenCV • CNN model building with keras

Train a Binary Classifier

In this liveProject, you’ll explore a pre-made dataset of meteorological records. You’ll learn the shape, size and type of data at hand, and discover factors that affect rainfall. You use scikit-learn and logistics regression to make initial predictions about future rainfall, evaluate their accuracy, and visualize emerging patterns using Seaborn and Matplotlib.

skills learned: manipulate data and conduct exploratory analysis • input missing values • engineer outliers • evaluate predictions

Train a CNN

You’re a data scientist at EKKo Inc., a machine learning consultancy that’s working on an embedded system to help deaf or hard of hearing people participate in online meetings and events on their mobile devices. To help the system recognize the hand gestures that represent the letters in American Sign Language (ASL), your task is to classify them using a convolutional neural network (CNN), an algorithm widely used for image processing applications.

You’ll write a Python script that preprocesses the ASL dataset, ensuring the model can interpret it. Using TensorFlow, a popular choice for such tasks, you’ll create and configure the CNN model. You’ll train the model and improve its performance by adding regularization to avoid overfitting, fine-tuning the learning rate (LR) to increase training speed, and introducing callbacks to monitor the training process. When you’re done, you’ll have firsthand experience using TensorFlow tools to configure various CNN hyperparameters, train a CNN onto an embedded board, and generate predictions from the CNN.

skills learned: preprocess the data • create and configure a CNN Model • use regularization to prevent overfitting • use learning rate (LR) to speed up model training• model training and introducing Callbacks

Train a DL Model

In this liveProject, you’ll create and train your deep learning model to predict where a new Airbnb listing will be priced above or below average pricing. You’ll put pandas, Keras, and scikit-learn into action, and master key deep learning techniques for tracking model performance. You’ll learn to harness callbacks in the Keras deep learning framework to make model training more efficient, and how to ensure consistent results in multiple training runs.

skills learned: use Keras to interface with TensorFlow • utilize Keras callbacks • track and assess model performance

Train a Logistic Regression

You’re a data scientist at EKKo Inc., a machine learning consultancy that has won a contract to create an embedded system that can be implemented on embedded systems. This system will enable people who are deaf or hard of hearing to participate in online meetings and events. After setting up an environment for running ML programs on an embedded system, you’ll process data from the existing American Sign Language (ASL) dataset and build, train, validate, and analyze the performance of a logistic regression model. By the end of this liveProject, you’ll be able to install, configure, and deploy a basic logistic regression algorithm that’s capable of classifying the letters of the ASL image dataset.

skills learned: hardware and software environment setup • basics of how neural networks function • data preprocessing and splitting for training, validation, and testing

Train a Model

In this liveProject you’ll use human pose data and TensorFlow.JS’s PoseNet to build and train a machine learning model that can recognize workout exercises. This model will record and recognize the workout session of a user, to be logged for future comparison. You’ll need to prepare data structures for the TensorFlow.js Dataset API, and execute one-hot-encoding with a simple mapping function. You’ll also define model architecture using the TensorFlow.js sequential API, and train your model using the fitDataset method. You’ll finish up by saving your trained model in local browser storage to allow for model reuse.

skills learned: creating new modules within a React app • cleaning up the data • preparing data for training

Train a Supervised Learning Image Classifier

In this liveProject, you’ll use the popular deep learning framework PyTorch to train a supervised learning model on a dataset of melanoma images. Your final product will be a basic image classifier that can spot the difference between cancerous and non-cancerous moles. You’ll create a custom dataset class and data loaders that can handle image preprocessing and data augmentation, and even improve the accuracy of your model with transfer learning.

skills learned: PyTorch for deep learning on the GPU • image classification using deep convolutional neural networks • transfer learning to improve model accuracy

Train and Score with Feature Store

In this liveProject, you’ll train your model and build a scoring pipeline using an ML feature store. You’ll explore a sample data set for diagnosing diabetes, generate new features and store them in a feature store, train and retrain ML models, and build a scoring process. You’ll employ common feature engineering techniques to train the model, then test and retrain it as needed. You’ll also work on setting up a scoring pipeline, and brainstorm ML development using a feature store. In this project, you will learn how to store the features for a machine learning model so they can be reused in other machine learning projects.

skills learned: generate new features and store them in a feature store • train and retrain ML models • build a scoring process

Train and Score with Raw Data

In this liveProject, you’ll train and evaluate a machine learning model for diagnosing diabetes, and set up a pipeline for your model to run effectively. You’ll start by exploring sample data, processing features, and performing common feature engineering techniques for treating outliers or missing data. After dividing your dataset into training and testing data, you’ll train a logistic regression model using scikit-learn. You will then retrain the model with a different set of features. Finally, you’ll pick a model for scoring and build a scoring pipeline. You will test your scoring process on a scoring dataset.

skills learned: classification with logistic regression • ML pipelines with scikit-learn • generating model pickle files with Joblib

Train on Microcontroller Devices

EKKo Inc., a machine learning consultancy, is working on an embedded system to enable deaf or hard of hearing people to participate in online meetings and events on their mobile devices. Your task as a data scientist is to optimize this system. Using the model optimization toolkit for TensorFlow, you’ll train a CNN model with an existing American Sign Language (ASL) dataset and convert it to TFLite format to reduce the code footprint. You’ll optimize it further using quantization, drastically reducing size and CPU consumption while maintaining model accuracy. Using batch normalization to decrease the number of training cycles, you’ll significantly speed up the CNN model’s training process. Lastly, you’ll integrate the quantization changes you’ve made by compiling and training the CNN model. When you’re finished, you’ll have a fully functional quantized CNN that can be run successfully on an embedded device.

skills learned: post-training quantization using tf-dataset • quantization awareness training and JIT • batch normalization for quantized model

Training and Deploying an ML Model as a Microservice

In this liveProject, you’ll fill the shoes of a developer for an e-commerce company looking to build a machine learning solution to help identify bad product reviews. If one of your company’s products receives too many bad reviews, it’s policy to take it down from the e-commerce store. Until now this process has been manual—but your boss has decided that this is too expensive and time-consuming.

Your mission is to automate this process, dramatically increasing the speed of identifying bad reviews, and decreasing the cost to your company. To complete this project you will have to train a machine learning model to recognize and rank positive and negative reviews, expose this model to an API so your website and partner sites can benefit from automatic rankings, and build a small webpage that can run your model for demonstration.

Updated March 2022

• Revised solutions for each milestone
• Added click-to-reveal help boxes with hints and guidance
• Added recommended books and learning resources
• Edited to focus on Amazon Web Services
• Added material on using AWS Lambda to support machine learning
• Clarified and simplified project prerequisites

skills learned: use an existent sentiment classification API • create a web API to expose an ML model • build a simple demo website

Transfer Learning

In this liveProject, you’ll take pretrained VGG16 and ResNet models from the Python Keras library and train them further upon your medical image dataset of X-ray and CT scans. This transfer learning is a highly effective technique for quickly generating reliable machine learning models when you only have a small data set. You’ll experiment with the Keras loss functions to determine which are best for COVID image classification, and check your training and prediction times as a critical parameter of real-world applications.

skills learned: use transfer learning for training VGG16 and ResNet models in Keras • deploy VGG16 and ResNet models for training on DICOM images • train VGG16 and ResNet models on two different types of medical image datasets

Transfer Learning with Transformers

In this liveProject, you’ll use transformer-based deep learning models to predict the tag of Reddit subreddits to help your company understand what its customers are saying about them. Transformers are the state of the art, large-scale deep language models pretrained on a huge corpus of text, and are capable of understanding the complexity of grammar really well. You’ll train this model on your own data set, and tune its hyperparameters for the best results.

skills learned: model monitoring • transfer learning using state-of-the-art NLP model • model diagnostics

Translation

In this liveProject you’ll develop a chatbot that can translate user messages, using the Hugging Face NLP library. Your challenges will include building the task with the T5 transformer, and build a Translation task considering different languages with mBART. You’ll classify the language of users' messages, and integrate your translation task with a chatbot.

skills learned: using a transformer to create a translation tool • build a chatbot using Hugging Face library to translate user messages from foreign languages ​​to English

Two Towers with TensorFlow Recommenders

Once you’ve handled the data, the real magic can begin! In this liveProject, you’ll implement a basic recommendation system using the TensorFlow Recommenders framework—designed specifically for this purpose. First, you’ll calculate four baselines that your future models will have to beat. Next, you’ll learn the basics of developing a model using TensorFlow Recommenders, then design a simple, two-feature Two Towers model. Lastly, you’ll enhance this simple model by adding all the features you created in the previous liveProject while maintaining model performance that beats your four established baselines.

skills learned: discover useful baselines • compute baselines and set minimum requirements to future models • implement and train a recommendation system using the TensorFlow Recommenders framework

UI for a Search API with Flask and Bootstrap

In this liveProject, you’ll use Docker, the Docker compose command line, and Bootstrap to construct an easy-to-use UI for your search engine. You’ll work to set up a single node Elasticsearch cluster, before putting together your Bootstrap UI, and then create a customized Docker image of the Flask API that serves your transformers model and frontend.

skills learned: packaging and deploying with Docker • search and indexing with Elasticsearch • building APIs with Flask

Understanding a Pandemic

In this liveProject, your task is to help fight a (fictional) global pandemic. You’ll gain hands-on experience plotting in Julia as you estimate new cases per day. You'll understand the affect of several measures through plot-analysis and basic time-series forecasting. Finally, you'll deduce what the best measures are, determine and predict their effects, and report your findings to decision-makers.

skills learned: plot in Julia • visualize data • manipulate data with dataframes

Untargeted Attacks on Your Classifier

Play the villain! Your goal is to mislead an existing DL model into incorrectly predicting the pattern. First, you’ll load your dataset, learn its structure, and examine a few random samples using OpenCV or Matplotlib. Using NumPy, you’ll prepare your dataset for training. Then, it’s attack time: Using FGSM and PGD, you’ll generate malicious inputs for the model in an effort to predict any class other than the correct one. Finally, you’ll enlist NumPy again to evaluate the success ratio of your attacks.

skills learned: basics of untargeted adversarial attacks (FGSM, PGD) • create an attack generator (Adversarial Robustness Toolbox - ART)

Use Machine Learning to Detect Phishing Websites

In this liveProject, you’ll take on the role of a data scientist employed by the cybersecurity manager of a large organization. Recently, your colleagues have received multiple fake emails containing links to phishing websites. Phishing attacks are one of the most common—and most effective—online security threats, and your manager is worried that passwords or other information will be given to an attacker. You have been assigned the task of creating a machine learning model that can detect whether a linked website is a phishing site. Your challenges will include loading and understanding a tabular dataset, cleaning your dataset, and building a logistic regression model.

skills learned: work on tabular datasets with pandas and NumPy • build a logistic regression classifier • use machine learning for security

Using Graph Networks to Recommend Books to Customers

In this liveProject, you’ll jump into the role of a data scientist working for an online bookstore. Your boss wants you to build a new recommendation system to help the marketing team match customers with a book that suits their interests. As the backbone of this new system, you’ve decided to create a graph network that will plot and analyze the relationship data of your platform’s users. To do this, you’ll need to import your data into pandas and transform it into an edge list, then build a network from the list. Once that’s accomplished, you’ll visualize and analyze the list to check its accuracy before you present your results as a web application that’s easy for the non-technical marketing team to use.

For this liveProject, you’ll be provided with real customer data from manning.com!

skills learned: data manipulation and analysis with pandas • building networks with Python • visualizing and analyzing graph networks

Using Machine Learning

Now that you’ve tackled volatility modeling in a traditional way, in this liveProject, your employer has challenged you to uplevel your volatility modeling by taking a more dynamic, data-dependent approach. By the end of this project, you’ll have firsthand experience modeling volatility using support vector machines with different kernels, neural networks, and deep learning models. What’s more, you’ll have the skills to determine if the machine learning-based models outperform the traditional parametric models.

skills learned: volatility modeling usingvolatility modeling using using support vector machines with different kernels, neural networks, and deep learning models using support vector machines with different kernels • neural networks • deep learning models

Using Neural Networks

In this liveProject, you’ll follow research laid out in a groundbreaking paper and work with algorithms that can take the aesthetic style of one image and apply it to another. You’ll use convolutional neural networks and transfer learning to build a simple image classifier and implement a neural style transfer. You’ll use TensorFlow and Keras to build your networks, Matplotlib and keras-vis to visualize them, and scikit-learn to analyze your results.

skills learned: convolutional neural networks with TensorFlow and Keras • analyzing model performance with scikit-learn • visualizing filter and class maximizations with keras-vis

Using Oversampling

In this liveProject, you’ll go hands-on with supervised learning methods for anomaly detection. You’ll explore an imbalanced dataset of seismic activity. To balance this dataset you will utilize the SMOTE and ADASYN oversampling algorithms to both generate synthetic examples of the minority class and then compare performance using random forest, logistic regression and Naive Bayes binary classification algorithms.

skills learned: run SMOTE using the imblearn library • run ADASYN using the imblearn library • analyze data using scikit-learn

Using PyOD

In this liveProject, you’ll use scikit-learn and the PyOD library to build an unsupervised machine learning model for detecting hyperthyroidism. PyOD is a Python toolkit for detecting outlying objects in multivariate data. You’ll compare performance between four different anomaly detection methods on a specialized thyroid dataset: PCA, Clustering-Based Local Outlier Factor (CBLOF), Histogram-Based Outlier Score (HBOS), and KNN algorithms.

skills learned: anomaly detection through PCA • anomaly detection through Clustering-Based Outlier Factor • anomaly detection through Histogram-Based Outlier Detection • anomaly detection through KNN

Using PyOD and Ensemble Methods

In this liveProject, you’ll explore a dataset with more variables and use scikit-learn and the PyOD library to build an unsupervised machine learning model for detecting cardiac arrhythmias. You’ll develop an algorithm which will detect arrhythmias from device data like EEG, using the Locally Selective Combination in Parallel Outlier Ensembles (LSCP) algorithm. A LSCP model accepts input as various other algorithms, and can be used to set up detectors with different settings.

skills learned: load MATLAB data in Python • run anomaly detection algorithms in PyOD • use the LSCP algorithm in PyOD in order to detect anomalies

Using Traditional Finance Models

In this liveProject, you’ll play the part of a freelance consultant who’s been hired to assess a company’s financial risk. Using the traditional volatility modeling packages ARCH and GARCH, you’ll model the volatility of S&P-500 stock prices—a good proxy for the entire financial market—and measure model performance. Then, you’ll optimize the model parameters using information criteria such as Bayesian Information Criteria (BIC). When you’ve completed the project, you’ll have a solid understanding of the logic of these traditional models and be ready to apply it to other models.

skills learned: volatility forecasting in financial modeling using ARCH and GARCH • optimizing financial model parameters using BIC

Using Undersampling

In this liveProject, you’ll utilize undersampling techniques to balance out a seismic activity dataset. To balance this dataset, you will utilize the ClusterCentroids, NearMiss and CondensedNearestNeighbor algorithms to downsample the majority class. Then, the performance is compared using random forest, logistic regression and Naive Bayes binary classification algorithms.

skills learned: run ClusterCentroids using the Imblearn library • run CondensedNearestNeighbor using the Imblearn library • run NearMiss using the imblearn library

Using scikit-learn

In this liveProject, you’ll explore the basics of anomaly detection by analyzing a medical dataset using unsupervised learning. You’ll create a model that can determine whether patients referred to a clinic have abnormal thyroid function. To accomplish this, you’ll download and prepare your dataset, and then utilize scikit-learn to compare different anomaly detection algorithms to find the most effective. You are going to use Isolation Forest, the Local Outlier Factor (LOF), One-Class SVM and Robust Covariance.

skills learned: run robust covariance with scikit-learn • run isolation forest with scikit-learn • run the LOF algorithm with scikit-learn

Variables

In this liveProject, you’ll help out a new Python developer with some mysterious bugs in their code. To unravel their issues, you’ll explore two different models for the behavior of Python variables and write some code that will either support or refute those models. This will help you reason about how variables behave in Python in regards to both mutable and immutable objects.

skills learned: understanding variables and whether they are containers or labels • using the id() function

Web-scraping for Text Threads

In this liveProject, you’ll harvest customer opinions about your company’s products from the comments left on the subreddit for your company, and store them in a database for future analysis. You’ll connect to the Reddit API, identify and clean up the data fields you need, and store the data into MongoDB.

skills learned: clean and filter for necessary data from JSON • save data into MongoDB in an organized manner • query the database for simple analytic tasks

When to Tweet for Best Effect

In this liveProject, you’ll build a fun (and useful!) data analysis tool that can determine which day of the week is the best to Tweet. You’ll test the hypothesis that Friday is the best day for engagement by calculating the p-variables and interpreting the results. You’ll utilize common techniques such as the permutation test and bonferroni correction to see if your hypothesis is accurate—essential skills for any data scientist.

skills learned: using Python fundamentals to set up environments to test hypotheses • using Pandas and NumPy for for data operations • doing permutation test for calculating p-values

Win a Card Game

In this liveProject, you’ll stretch your Python data science skills by building a simulator for popular Blackjack variant game 21. You’ll design your simulation step-by-step, then use visualization techniques to interpret your findings. By the end of your project, you’ll have a winning strategy for playing card games and new skills with fundamental Python libraries like NumPy and SciPy.

skills learned: simulating real-life game environments in Python • using NumPy and SciPy for mathematical operations • visualizing results with Matplotlib and seaborn

20 Questions

In this liveProject, you’ll use Python to create a command-line version of the classic guessing game 20 Questions. In your version of the game, the computer will have up to twenty chances to guess what animal the player is thinking of. You’ll use Jupyter notebooks to code your program and define functions that build a knowledge base in real time. Then, you’ll test and debug your code by playing against the computer.

skills learned: data processing • user input • randomness • data filtering • CRUD operations • modularizing code • debugging

2D Plotting with Matplotlib

ABC Bikes Inc. is considering launching a bike rental service in your area. As the company’s data analyst, your task is to provide decision-driving insights on bike rental trends. You’ll extract the relevant data from a publicly available dataset into a pandas data frame. Then, using Python plotting library Matplotlib, you’ll create line plots to visualize changes for bike rentals on a single day as well as over a specific time period, create a grouped bar plot to visualize a comparison of renting data for two years, create subplots with different chart types, and create a violin plot to visualize renting patterns over four seasons. When you’re done, you’ll know how to use Matplotlib to create accurate and informative 2D visualizations.

skills learned: use import statement to load required libraries and modules into Jupyter Notebook • use pandas functions to read data from CSV files into a data frame object • use pandas functions to subset required rows and columns for plotting • use Matplotlib API/functions to create, customize, and save plots

3D Medical Image Analysis with PyTorch

In this liveProject, you’ll take on the role of a machine learning engineer at a healthcare imaging company, processing and analyzing magnetic resonance (MR) brain images. Your current medical image analysis pipelines are set up to use two types of MR images, but a new set of customer data has only one of those types! Your challenge is to build a convolutional neural network that can perform an image translation to provide you with your missing data. You’ll do this using the deep learning framework PyTorch and a large preprocessed set of MR brain images. The company also wants to make sure your image translation convolutional neural network reliably produces the desired MR image, so you will need to provide qualitative and quantitative results demonstrating your method’s effectiveness.

skills learned: train a neural network for a regression task • build a CNN • handle and visualize medical imaging data

3D Plotting with Matplotlib

ABC Bikes Inc. is considering expanding its bike rental service to a new county. Your job, as the company’s data analyst, is to determine factors that impact the demand for rental bikes in that area. You’ll extract the relevant data from a publicly available dataset into a pandas data frame. Then, using Matplotlib’s mplot3d toolkit, you’ll plot 3D graphs to simultaneously visualize more than two data features, enabling you to determine useful patterns including how temperature, time of day, and month of year impact bike rentals. When you’re finished, you’ll know how to use Matplotlib to create, customize, and rotate your 3D plots to gather useful and interesting insights.

skills learned: visualize on 3D scatter plot and 3D surface plot • interactive visualization

AWS Athena

QryptoTremolo, a startup that’s developing a set of next-generation financial services, has hired you to help upgrade its security. The company has set up an organizational AWS CloudTrail, which deposits all the events of QryptoTremolo’s AWS control plane, as well as some data plane, in an S3 bucket within the security account. But the events are stored in a form that doesn’t lend itself well to forensic queries. QryptoTremolo has chosen AWS Athena as a query-friendly solution. Your task is to configure AWS Athena to allow for sending advanced forensic queries to an S3 bucket filled with organizational CloudTrail events. You’ll learn how to create forensic SQL queries and issue them in a performant way across all accounts in the organization.

skills learned: Athena • AWS CloudTrail • Boto

Add Background to a Green Screen Video

In this quick liveProject, you’ll develop a low-resource way to add a background to a green screen video using thresholding. You’ll read the frames of a green screen video, perform thresholding to separate the green screen background, apply the new background image, and write the revised frames to a new video.

skills learned: reading and writing videos • using binary thresholding

Add Knowledge to the Chatbot

In order for the Llama-based chatbot app to answer chemistry questions reliably and without hallucination, you need to ground and supplement it with facts and knowledge. That’s why your local school district has tasked you with using RAG (Retrieval Augmented Generation) to help improve the capabilities of the chemistry chatbot app. RAG allows an LLM to search a knowledge base to help answer questions, avoiding unfortunate hallucinations. You’ll create a vector database for chemistry textbooks as the knowledge base, establish an RAG API server for the chatbot, and then deploy your new bot to both the web and Discord.

skills learned: Create embeddings from a body of text • store embeddings in a vector database • build a RAG LLM application • create a Discord chatbot

Adversarial Training

Protect your model by implementing adversarial training, the easiest method of safeguarding against adversarial attacks. You’ll load your dataset, learn its structure, and examine a few random samples using OpenCV or Matplotlib. Using Numpy, you’ll prepare your dataset for training, then you’ll use FGSM to generate malicious input for both untargeted and targeted attacks on a trained DL model. For each type of attack, you’ll evaluate your model before and after you apply adversarial training-based mitigation methods, gauging the success of your defense.

skills learned: basics of adversarial training · iterative retraining with Keras • CleverHans attack generator

Algorithmic Trading with Machine Learning

Machine learning and the growing availability of diverse financial data has created powerful and exciting new approaches to quantitative investment. In this liveProject, you’ll step into the role of a data scientist for a hedge fund to deliver a machine learning model that can inform a profitable trading strategy.

You’ll go hands-on to build an end-to-end strategy workflow that includes sourcing market data, engineering predictive features, and designing and comparing various ML models. Throughout the liveProject you will work with libraries and tools from the industry-standard Python data ecosystem. You’ll tackle challenges such as training a regularized linear regression model, tuning a gradient boosting ML model, and evaluating the performance of your strategy—all essential skills for success in this highly lucrative area of machine learning.

skills learned: engineer financial features • build and test fundamental and advanced ML models to predict asset returns with scikit-learn and LightGBM • develop a trading strategy by defining rules that translate model predictions into trades

Analysis and Feature Extraction

Feature extraction is an essential part of detecting deepfake videos. In this liveProject, you’ll analyze both deepfaked and real video data in order to determine what features are common in faked videos. You’ll then compute those features for faces detected in the videos to determine which are fake and which are real.

skills learned: understand the differences between deepfake faces and real faces • compare images and their histograms

Analyze Reports with Hugging Face

You’re a data scientist at Finative, an environmental, social, and governance (ESG) analytics company that analyzes a high volume of data using advanced natural language processing (NLP) techniques in order to provide its clients insights for sustainable investing. Recently, your CEO has decided that Finative should increase its own financial sustainability. Your task is to classify sustainability reports of a publicly traded company in an efficient and sustainable way.

You’ll learn the fundamental mathematics—including backpropagation, matrix multiplication, and attention mechanisms—of Transformers, empowering you to optimize your model’s performance, improve its efficiency, and handle undesirable model predictions. You’ll use Python’s pdfplumber library to extract text from a sustainability report for quick delivery to your CEO. To further increase efficiency, you’ll save training time by using a language model that’s been pre-trained with ESG data to build a pipeline for the model and classify the sustainability report.

skills learned: understand the attention mechanism (Transformers) • build a model pipeline with Hugging Face • extract and prepare data from PDF files

Analyze a Trust Network

In this liveProject, you’ll turn your data science skills to analyzing an OTC network dataset scraped from bitcoin users in order to establish the most (and least!) trustworthy users. You’ll analyze a provided graph dataset, visualize it, generate features, and then create user clusters. You’ll start out by reading and examining the trust network dataset in Python, then create and interpret user clusters, and finally visualize the nodes and edges of the network dataset. This fast and engaging data science project will stretch your skills and build your knowledge of clustering.

skills learned: pandas for data operations • scikit-learn for clustering • NetworkX to analyze and visualize network datasets

Analyze and Cluster OpenStreetMap Data

In this liveProject, you’ll use LynxKite and graph data techniques to identify some of the most important geographic points in the city of Bruges. You’ll start by downloading and processing map data, and then use a simple Python program to convert it into a graph. You’ll use graph centrality metrics to quantify the importance of vertices in your graph, and determine some of Bruges’ important locations. You’ll then use the same structure to figure out the main areas of the city without using actual district data.

skills learned: convert map data to a graph representation • visualize graphs • compute and use common centrality metrics

Analyzing GitHub Data with GCP Services

GitHub is stuffed full of free and open-source development tools, all with no guarantee that they’re stable, up-to-date, or even still maintained! Companies need a way to ensure that the GitHub projects they’re adopting come with a clean bill of health — and for that, they turn to you!

You’re the CTO of GitHub Health, a unique startup that provides reports and analysis of GitHub software projects. But your findings aren’t based on gossip, they’re based on data. In this liveProject, you’ll build a serverless data system that can extract meaningful data from GitHub, store it in a database, and display the statistics using Google Cloud. Once you’re done, you’ll generate a report for a new client on which of three open-source projects is the best choice for its new DevOps team.

skills learned: serverless data extraction • storage and presentation within Google Cloud

Apply a Face Mask to a Selfie

In this quick liveProject, you’ll develop a selfie enhancement application to build awareness of the importance of mask-wearing during a pandemic. You’ll use a laptop webcam to capture a user’s selfie, load a face mask image with a transparent background, and apply the face mask onto the image at the location selected by the user.

skills learned: tracking mouse events • using bitwise operations • saving images

Arbitrage Pricing Model

In this liveProject, you’ll build a machine learning system based on the Arbitrage Pricing Theorem (APT) that can create a diversified investment portfolio designed to avoid risks. APT allows you to model the effects of different scenarios on an investment portfolio, and you’ll test this theorem on a portfolio of social media companies. You’ll run analysis to estimate coefficients, and conduct sensitivity analysis to determine important macroeconomic factors. Finally, you’ll interpret your results to find out what issues your portfolio is the most sensitive to.

skills learned: risk management • multi-factor modeling • multivariate linear regression

Augment Training Data and Classify Text

In this liveProject, you’ll augment text-based training data for a sentiment analysis algorithm with artificially generated positive reviews. You’ll merge the synthetic positive reviews with an unbalanced dataset focused on negative reviews, thereby creating a balanced dataset for your model to train on. You’ll train your model, then evaluate its metrics using sklearn.

skills learned: merging training and synthetic data • building and training a text classification model • scoring text data with the model

Augment with Embeddings

In this liveProject, you’ll become a software engineer at InfoHub, an up-and-coming AI startup looking to revolutionize how companies interact with their knowledge bases. InfoHub seeks to utilize groundbreaking large language models to deliver a system whereby a user’s questions about company data can be answered through a Q&A-style language interface. You’ll begin by assisting them in creating this tool by processing, tokenizing, and converting data into embeddings with BeautifulSoup and tiktoken.

skills learned: use of embeddings with the OpenAI API for similarity search

Authorship Identification with Text Mining and Machine Learning

In this liveProject, you’ll step into the boots of an investigator trying to find the anonymous author of a seriously defamatory blog post. You’ve narrowed down your list of suspects, acquired a dataset of writing samples, and now plan to find the culprit using a custom machine learning project. Your challenge is to build an authorship analysis model that will match a sample to the defamatory blogpost and reveal the guilty party. To do this, you’ll need to extract data from a corpus of documents, build a model that can learn authorship style, scale the model to handle hundreds of suspects, and finally develop a user-friendly program that will allow non-technical colleagues to make use of your findings.

skills learned: extract features from text using scikit-learn and spaCy • build a predictive classification model • visualize authorship styles with an interactive plot • incorporate your trained model into a user-friendly program

Automate Data Preprocessing

In this liveProject, you’ll utilize automated machine learning tools to help the data preprocessing and feature engineering for creating an image classification model. You’ll start with the basics of data preprocessing, and then see how useful AutoML solutions can make this process quicker and easier. The rewards are big, as properly preprocessed data can dramatically improve the outcomes of a deep learning project.

skills learned: implementing data preprocessing for image data • training deep learning models adopting the data preprocessing

Automate Pipeline

In this liveProject, you’ll implement an AutoML pipeline using the AutoKeras functional API. You’ll make use of the built-in blocks in AutoKeras to conduct automated hyperparameter tuning and model selection for creating an image classification model. Your challenges will include customizing both a sequential AutoML pipeline, and customizing graph-structured AutoML Pipeline.

skills learned: implementing AutoML using the automated data preprocessing API and AutoKeras Task API • customizing sequential and graph-structured AutoML pipelines

Automate Reports

As a member of the development team at Piper Data Concepts, you’ll carry out the final steps of a workflow-improvement project: enabling your client’s staff to gather workflow process information in real-time. Several prototypes have been built, and the client’s workflow is more resilient than ever. You’ll write Python code that consumes messages from Kafka and prepares them for storing in the database, create Postgres queries to access the aggregated data, and build reports in CSV files to be read by visualization tools—and ultimately, your client’s staff. When you’re done, your client’s modern system will provide a feedback loop, enable external API access to status updates, and be ready for more specialized services to be plugged in later, with no code changes.

skills learned: end-to-end pipeline • Kafka data into the database • web scraping • creating reports

Automatically Tracking File Changes with Python

In this liveProject, you’ll use the flexible Python language and SQLite database to develop a simple-but-useful document tracking tool for a pharmaceutical company. This sort of automation tool is in high demand from businesses that need to keep track of changes to their important documents. Your challenges will include creating and connecting to a local SQLite database, working with file information and MD5 hashes, and applying automation principles to your tool.

skills learned: walking directory trees • identifying files • calculating file MD5 hashes • working with SQLite • creating Excel reports with the openpyxl library

BB84 Key Distribution Protocol

In this liveProject, you’ll take on the role of a cryptography expert evaluating quantum computing for the purpose of key distribution. Key distribution generates a random secret key which is then shared between only two parties, and is a crucial part of many cryptographic algorithms. You’ll work to implement the common BB84 key distribution, simulate using the BB84 protocol to generate a shared key, and then visualize the steps of the BB84 protocol.

skills learned: designing and writing simple Q# code • running simulations of quantum code using a full state quantum simulator • writing classical code in C# or Python that calls quantum code

Basic API of AutoKeras

In this liveProject, you’ll learn how to use the Task and Functional API of Keras to build an automated deep learning model for image classification. This AutoML approach will allow you to avoid time spent selecting and tuning your deep learning algorithms. You’ll work with the detailed CIFAR dataset of animal images which is easy to access through the TensorFlow Keras API.

skills learned: task and Functional API of AutoKeras • training deep learning models automatically with AutoKeras

Basic Deep Learning

In this liveProject, you’ll learn and apply some of the basics of deep learning in order to build the foundations of an AutoML image classifier. You’ll discover the basic deep learning models for image classification, and then experiment with tuning the hyperparameters of a convolutional neural network to improve your results.

skills learned: constructing deep learning models for image classification • search the hyperparameters of deep models with grid search and random search

Bayesian Dynamic Linear Modeling

In this liveProject, you’ll build a Bayesian dynamic linear model that can take account of sudden state space changes and rapidly react to dramatic trend changes. These trend changes could take many forms—from heightened demand during a major sporting event, to a global pandemic that causes cancellations to skyrocket. You’ll use the PyDLM library to generate forecasts that can dynamically adapt to the unforeseen, and quickly shift to making accurate predictions for a new reality.

skills learned: build a simple DLM model • build Bayesian dynamic linear models with the PyDLM library

Bayesian Statistical Methods with PyMC3

In this liveProject, you’ll use PyMC3 to generate a posterior distribution of hotel cancellations. This will allow you to build a predictive model that can update its predicted probabilities by incorporating new information. You’ll master methods for modelling mean and standard deviations based on the selected prior values, generating distributions to determine if data follows an AR(1) process, modeling of stochastic volatility and generalized linear modelling with PyMC3.

skills learned: manipulating a dataset with pandas • probabilistic time series analysis with PyMC3 • generating posterior distributions • modeling autoregressive processes

Boost Specific Results

Help Sneakers to the Max’s customers find the perfect sneakers! As a software engineer at the e-commerce company, your task is to improve its search engine—and hopefully sales—by fine-tuning the order of search results. You’ll adjust the index mapping to import data about sneaker release dates, popularity, and in-stock status. Then, you’ll write extensions to the Elasticsearch queries that boost recent and popular results, forcing out-of-stock sneakers lower. Finally, you’ll use everything you learned to return the most relevant sneakers using a judgment list provided by the product managers. When you’re finished, you’ll have the skills to enhance Elasticsearch queries, while Sneakers to the Max’s customers will easily find the perfect pair.

skills learned: analyze the results of different boosting mechanisms and tune them to work together

Build Network Models for Pandemics

In this liveProject, you’ll use machine learning to construct a contact tracing network for COVID-19 using location recordings from smart phone data. You’ll read the location of infected individuals, and generate a contact network of individuals who have been within two meters. Once you’ve established this tracing system, you’ll implement a distributed algorithm that can compute the average infection rate for each connected component of the contact network.

skills learned: reading in location recordings from a CSV file • determining geodesic distances between locations • implement a simple algorithm that computes a summary statistic of networked data

Build a ResNet Model

In this liveProject, you’ll build a ResNet deep learning model from scratch to analyze medical imagery. A ResNet is a deep neural network model which uses "Residual blocks" and "skip connections" to reduce the need for very deep networks while still achieving high accuracy. You’ll then train your model on X-ray and CT datasets, and plot validation loss, and accuracies vs. epochs. You’ll build an important familiarity with the functional blocks of a DL model, how data must be formatted, and which layers to use to solve your problems.

skills learned: build a ResNet deep learning architecture with basic functional components in Keras • train ResNet model hyperparameters on two different types of medical image datasets (X-ray, CT) • tune ResNet model to improve performance

Build a Self-Serve Data Platform

You’re a consultant working for Messflix Inc., a movie and TV-show streaming platform. Your task is to set up a Python prototype implementation of the data mesh to roll out the technical components of a data mesh. Using Python and pandas, you’ll write a Python function that creates an empty CSV file with the predefined attributes of your data product, builds a data catalog by creating Python functions that write to the CSV file, and sets up standardized access to the CSV datasets. When you’re done, you’ll have hands-on experience building a minimal self-serve data platform using simple techniques.

skills learned: build a self-serve data platform • build a small data catalog • consume data inside a data mesh • store data products

Build a VGG16 Model

In this liveProject, you’ll build a VGG16 deep learning model from scratch to analyze medical imagery. A VGG16 is a deep convolutional network model which has shown to achieve high accuracy in image based pattern recognition tasks. You’ll then train your model on X-ray and CT datasets, and plot validation loss, and accuracies vs. epochs. You’ll build an important familiarity with the functional blocks of a DL model, how data must be formatted, and which layers to use to solve your problems.

skills learned: build a VGG16 deep learning architecture in Keras • use custom image data generators in Keras • train VGG16 model on two different types of medical image datasets (X-ray, CT) • tune VGG16 model hyperparameters to improve performance

Build an Extensible CLI with Python

In this liveProject, you’ll take on the challenge of designing and building a custom CLI using Python. Command line interface tools automate repetitive tasks and help streamline software development. Companies that use bespoke CLIs can easily manage change, smooth friction between teams, and reduce time spent on red tape.

You’ll join up with KlickBrick, a toy company whose digital expansion is grinding to a halt under mismanagement, and create a CLI framework that helps bring development teams together. You’ll need to assemble the infrastructure for your CLI, then develop functionality for onboarding employees, initializing new software project repositories, and reporting CLI usage metrics to a server.

skills learned: implementing an extensible CLI tool • translating business requirements into Integration Tests using the Behave framework • real-world software engineering practices for building a high-quality and maintainable software project

Build an ML Recommender System

Recommender systems are one of the most popular and lucrative uses of machine learning, allowing businesses and organizations to give personalized suggestions to their customers.

In this liveProject, you’ll use common tools of the Python data ecosystem to design, build, and evaluate a movie recommendation model for the movie website. You’ll kick off your project by building simple genre charts, then work with existing movie rating data to implement personalized recommendations for each customer. When you’ve completed this hands-on and interesting project, you’ll have mastered a cornerstone technique of machine learning that’s in demand across companies and industries.

skills learned: load and analyze a dataset • train a machine learning model • evaluate a machine learning model • use a trained ML model on a website • collaborative filtering

Build and Deploy the Chatbot

In this liveProject, you’ll step into the role of a software developer at InfoHub, an AI startup attempting to build a chatbot that can answer queries about a company’s knowledge base. You’ve already built the basics of the chatbot—now it needs to be deployed! To do this, you’ll use the LangChain feature LangServe to create a LangChain server application. You’ll build your chatbot using Streamlit and then deploy it to the Streamlit cloud.

skills learned: create a chatbot using LangServe and Streamlit • deploy to the cloud through Streamlit Cloud

Building Domain-Specific Language Models

In this liveProject, you’ll step into the role of a natural language processing data scientist working for Stack Exchange. Stack Exchange runs a network of question-and-answer sites on diverse topics ranging from programming to cooking. Your boss wants you to create language models that are tuned to the statistical, probabilistic, and technical jargon present in different Stack Exchange sites.

Language is domain-specific—an insurance company’s documents will use very different terminology than a post on a social media site. Because of this, off-the-shelf NLP models trained on generic text can be inaccurate for specialized domains such as healthcare, legal, clinical, and agricultural language. Your goal is to build a language model capable of query completion and larger text generation for Stack Exchange sites. At the end of this project, you will be able to build the foundations of any domain-specific NLP system by creating a robust and efficient language model using statistical and deep learning techniques.

Updated: March 2022

• Fully updated to the latest version of AllenNLP
• Improved GPU compatibility for training larger models
• New help layers with detailed hints and guidance
• New preprocessing steps for data preparation
• Adjusted prerequisites and libraries

skills learned: data manipulation with NumPy and pandas • text preprocessing with NLTK • train an RNN with PyTorch • score and evaluate language models

Building a Search API with Elasticsearch and BERT

In this liveProject, you’ll bring together multiple tools and models to construct a production-grade smart search engine. You’ll combine off-the-shelf Elasticsearch models for keyword search with your own semantic search API using transformers. Start by setting up and indexing an Elasticsearch Docker container, then quickly move on to boosting search relevance with BERT. Finally, you’ll set up a Flask API to serve a BERT model and look into customizing your search engine for your own particular topics of interest.

skills learned: Packaging and deploying with Docker • search and indexing with Elasticsearch • building APIs with Flask

Building an ML Pipeline with Kubeflow

Putting machine learning into production can often be a complex task. The Kubeflow platform helps streamline this process with simple and scalable ML workflow deployment. In this liveProject, you’ll put Kubeflow into action to help your team roll out their new license plate recognition deep learning system.

You’ll help data scientist colleagues by standardizing their working environment, and automating away many tedious and error-prone tasks. Your challenges will include restructuring a complex deep learning project to make it Kubeflow-friendly, and developing reusable components that can be transferred to other machine learning pipelines.

skills learned: create a machine learning pipeline that is composable and scalable • structure a non-trivial ML project to make it Kubeflow-friendly • view training runs in Tensorboard • use Kubeflow Metadata to capture and locate generated data

CI/CD Application Testing with Jenkins X

Jenkins X helps supercharge your software delivery with both a CI/CD toolkit and isolated preview environments that let you test how your whole platform behaves with new features. In this liveProject, you will implement continuous integration and continuous deployment workflows by adding Jenkins X.

You’ll step into the role of a software engineer at Money FX, a Fintech company whose success rests in its speedy payment processing platform. You’ve been assigned the task of implementing a new strategic way to run platform tests as part of a CI/CD process, and you’ve decided to use Jenkins X to make this task easier. Your challenges will include moving the current service to Kubernetes, introducing Jenkins X to the testing process, validating the new workflow with code addition, and extending Jenkins X with a new pipeline.

skills learned: Jenkins X • CI/CD • Kubernetes • GitOps

CI/CD and Change Control

You’ve been hired by QryptoTremolo, a startup that’s been developing a set of next-generation financial services, to lower the risk of security breaches. The company has chosen to develop and operate in AWS. Its development teams have had unrestricted access to the AWS accounts, and ad hoc development has evolved organically, increasing the risk for security breaches. To uplevel security in these accounts, you’ll deploy CloudFormation templates and develop a suite of automated pytest tests to verify their configuration. Next, you’ll build a CI/CD CodeBuild pipeline that triggers on infrastructure changes and deploys and tests the changes across the organization. This change control automation will enable you to respond quickly—and confidently—to changing business needs.

skills learned: AWS CodeBuild • AWS CodeCommit • StackSets • AWS CLI • pytest • CloudWatchEvents

Capital Asset Pricing Model

In this liveProject, you’ll build a machine learning system based on the Capital Asset Pricing Model (CAPM) that can determine if certain stocks are over or undervalued, and the risk level of these stocks relative to the market index. CAPM is a powerful tool in finance due to its intuitive and easy-to-apply nature, and in this project you’ll use it to estimate coefficients, determine valuation accuracy, and finally find which stock promises the best risk-return relationship.

skills learned: risk management • valuation • univariate linear regression

Centralized Logging

You’re an enterprise architect for Padre Inc., which has acquired Tiddler Inc., an insurance analytics startup, for its SaaS platform. Since your company runs its cloud operations on AWS and the startup uses Google Cloud Platform, your task is to manage multi-cloud operations. As part of that effort, you’ll implement centralized logging to simplify log analysis across multiple applications. After exploring centralized logging using the ELK stack, you’ll create a centralized logging instance in Google Cloud Platform and stream the logs from AWS to Google Cloud. To signal the need for corrective actions, you’ll use Kibana to run queries against the log data and create an alert based on log events. When you’re done, you’ll have created a centralized logging flow that improves site reliability.

skills learned: Elasticsearch • Logstash • Kibana • AWS Cloudwatch • AWS Lambda

Chatbot with Llama

In this liveProject, you’ll take on the role of a full-stack developer working for a school district. Your district wants to further student learning by developing its own large language model (LLM) applications to assist students, parents, and teachers. Your goal is to develop an MVP of this by creating a chatbot that can answer questions and engage in follow-up conversations. You’ll utilize the open source Llama LLM from Meta AI to do this. Your model will need to run with very low resources—it needs to work on the school’s outdated computers—and be equipped with an intuitive user interface. Let’s get started!

skills learned: Quantize an open-source LLM • build and use prompts and prompt templates for a chatbot LLM • build a web UI for the chatbot

Churn Analysis

In this liveProject, you’ll analyze a data set of customers metrics to uncover patterns and behaviors that indicate customer churn. You’ll determine key details like rate of retention, and analyze your data to determine which metrics are associated with customers leaving and customers staying loyal. You’ll analyze how customer behaviors are related to one another, and see if you can expand on your datasets functionality to find new metrics that indicate churn.

skills learned: create a data set from raw data stored in a database • visualize how different customer metrics relate to the outcome of churn • analyze different customer behaviors

Churn Forecasting

In this liveProject, you’ll develop a simple machine learning algorithm that can determine from data metrics which of your customers are likely to churn out of an online business. You’ll develop an XGBoost machine learning model and explain its predictions, then build a logistic regression statistical forecasting and analysis model to also predict churn. By the time you’re done, you’ll have two reliable and automated tools for spotting when customers are likely to leave so that your marketing team can easily intercede.

skills learned: XGBoost machine learning model churn forecasting • logistic regression model churn forecasting • explain machine learning model results

Class Methods and Variables

In this liveProject, you’ll write a Python class to manage network connections to your server. You want your class to track the port number and the connections, but instance data is, by definition, not associated with your class. Your challenge is to explore and determine the best ways to associate data with a class and learn how to use a class method.

skills learned: write a Python class to manage network connections • explore ways to access class variables • create class methods

Classification with XGBoost

In this liveProject, you’ll use the Julia language to build a classification-based machine learning model that can predict the salary of a customer based on their sociodemographic data. This model will then be used to serve premium advertising to wealthier customers. You’ll build and evaluate XGBoost models with the dedicated Julia XGBoost.jl package, tune the hyperparameters, and assess your model’s capabilities using ROC curve, and measures such as AUC, accuracy, recall, and precision.

skills learned: comparing distributions of predefined train and test data • building XGBoost model in Julia • evaluating classification model’s quality

Cleaning and Exploring Text Data

In this liveProject, you’ll clean and analyze data scraped from Reddit to determine customer opinions of your products within a set time period. You’ll utilize common natural language processing techniques such as stemming, tokenization, and latent dirichlet allocation (LDA) to discover patterns in people’s opinions, and then visualize your results and summarize your findings.

skills learned: NLP preprocessing techniques such as stemming, n-gram, and removal of stop words using NLTK • visualize word cloud and top n-grams • perform topic modeling using LDA

CloudTrail Test and CI/CD

In this liveProject, you’ll ensure that Qrypto Tremolo’s security team can make incremental, verifiable changes to its AWS CloudTrail configuration. To achieve this, you’ll develop a suite of automated tests and CI/CD CodeBuild pipelines to deploy and test changes across the organization.

skills learned: develop automated tests to verify the AWS CloudTrail configuration • develop a CodeBuild pipeline that will trigger on changes to the infrastructure as code

Collaborative Similarities

In this liveProject, you’ll build a movie recommendation system based on movies that your users have previously interacted with. This system is designed to boost engagement and keep your users on your site. You’ll develop a competitive array of similarity functions, and create your own recommender system based on these similarities. Finally, you’ll evaluate your system’s effectiveness and tune its parameters.

skills learned: read, process, and exploit user-item data • define and compute similarities between users and items using interactions • compute similarities between users and items in a recommender system

Content-Based Similarities

In this liveProject, you’ll build a movie recommendation system based on the content and metadata of movies in your system. This system is intended to maximize the satisfaction of your movie-watching users. You’ll start with an analysis to determine the content of your movies, then use that data to implement content-based similarities for both products and users. You’ll build and evaluate your recommender system based on these connections, till it’s the best it can be!

skills learned: define and compute similarities between users and items using content and metadata • define recommender systems based on the definition of when two users or items are close

Create a Color Blender

In this quick liveProject, you’ll build a custom graphic design tool that can generate a matching RGB color palette for a chosen color. You’ll create a blank canvas using NumPy arrays, display images using OpenCV, track a user’s activity with mouse events and flags, and also handle keyboard input.

skills learned: creating NumPy arrays • displaying images using OpenCV • using mouse events and flags to track user's activity on display window • handling keyboard input

Create a Data Platform for Real-Time Anomaly Detection

In this liveProject, you’ll close the gap between “data analyst” and “software engineer” by building a working data platform. You’ll join up with AnomalousDex Inc., a startup that specializes in personalized end-to-end data products, and create a working prototype of their anomaly detection platform to showcase to prospective customers. This requires connecting up multiple cross-discipline components, from data science to systems management.

This platform consists of three principal components: a service that serves the anomaly detection data model, the modeling platform, and a dashboard visualization tool. Your challenge is to develop all of these features, going hands-on with software architecture, data engineering, microservices, and dockerizing. You’ll also dive into the essentials of monitoring and metrics, and even train an unsupervised learning anomaly detection model!

skills learned: design a data platform architecture consisting of several Dockerized components • train an anomaly detection model with scikit-learn and deploy it in a web service • monitor the web service and anomaly detection model with Prometheus and Grafana

Create a Neural Network

In this liveProject, you’ll use Keras to create a deep learning model for predicting basketball scores. Once your model is created, you’ll train it up on sample data and then validate your results to ensure it’s still accurate when applied to data from the real world.

skills learned: manipulate data with pandas • build a neural network with Keras • test and validate a neural network

Create a Product Index

E-commerce company Sneakers to the Max wants to strengthen its bottom line with an improved search engine that provides better and quicker results to customers. As a software engineer, your task is to create a structure for importing data into the search engine. You’ll write a Python script to import sneaker data into Elasticsearch. Next, you’ll create an index for the sneakers and use aliases to keep the data available while importing a fresh catalog. To verify your import and gauge the quality of your data, you’ll create a Kibana dashboard. Using Elasticsearch Bulk, you’ll improve the indexing performance of the sneaker data you need. When you’re finished, you’ll have created a complete structure for your imports that enables frequent, regular updates without downtime.

skills learned: import JSON objects from a file using standard Python language constructs • use Python built-in file IO mechanisms • interact with your Elasticsearch cluster using the Elasticsearch library for Python • verify if the import was run successfully using Kibana

Creating Features

In this liveProject, you’ll process raw data to make it ready for a machine learning model to diagnose diabetes rates. You’ll use feature engineering techniques to generate ML features from raw data. To make sense of your data, you will undertake data profiling, exploratory data analysis, analyze independent/dependent variables, and visualize data patterns. You’ll evaluate the correlation between dependent and independent variables to identify relevant features. You’ll even generate additional features as needed. Additionally, you will apply feature engineering techniques such as treating missing values and outliers to make your features ready for model training.

skills learned: use feature engineering techniques to generate ML features from raw data • data profiling, exploratory data analysis, analyze independent/dependent variables, and visualize data patterns • generate additional features

Cryptogram

In this liveProject, you’ll use Python to build a command-line game that produces a cryptogram to decode. You’ll build a simple web scraper to pull data from a quotation website, then generate an encoded puzzle for the player to decipher. You’ll interact with a web server to create your game, using the Python library BeautifulSoup.

skills learned: web scraping • error handling • basic encryption • modularizing code • requests • BeautifulSoup4 • debugging • exceptions

Custom Rule Development

Enter course descriptionA string of high-profile security breaches has been featured recently on the news, and QryptoTremolo doesn’t want to be the next headline. You’ve been hired by the startup, which is developing a set of next-generation financial services, to lower the risk of breaches and bring security controls to its AWS accounts. Leveraging pytest and Boto, you’ll develop and deploy a custom Config rule to look for peculiar misconfigurations or conditions that aren’t covered by AWS’s off-the-self managed rules.

skills learned: AWS Config rules • pytest • AWS Lambda Functions • Boto

Customer Behavior

In this liveProject, you’ll clean and process data from Manning’s liveBook platform and prepare it for investigatory analysis to discover causes of customer churn. You’ll load a raw data dump of customer behavior data into PostgreSQL for analysis, categorize and check the event records for bad data, and calculate a basic set of interpretable customer metrics. Some SQL skills and a little Python is all you need to produce a clean and easy-to-interpret data set of customer behavior.

skills learned: loading data into a database • design and calculate customer metrics • perform quality assurance analytics

Customize AutoKeras Blocks

In this liveProject, you’ll build an AutoKeras block for image classification to help tune a neural network and customize the search space. You’ll then combine the block you have created with other prebuilt AutoKeras blocks to put together a complete automated machine learning pipeline.

skills learned: customizing an AutoKeras block for CIFAR-10 image classification • designing a hyperblock for model selection

Data Augmentation for ML

As a machine learning engineer in an online recruiting tech company or HR department of a large organization, your task is to address a lack of data, a common problem in data science projects. To solve this, you’ll create multiple tools to augment processed data, increasing its volume and learning essentials about probability distributions, random sampling, and OOP. Completing this project will enhance your data analysis and visualization skills, taking you further down the path to a career in data science.

skills learned: data augmentation for ML • deal with missing data with statistical modeling • build data augmentation tools with OOP

Data Ingestion and Cleaning

Imagine you’re a data engineer working at an enterprise. In this liveProject, you’ll set up a Databricks platform, creating clusters and notebooks, interacting with the Databricks File System (DBFS), and leveraging important Databricks features. You’ll also gain first-hand experience with Apache Spark—the world’s most widely used distributed processing framework—on tasks like reading the input data in CSV and JSON format, filtering, and writing the data to the data lake’s curated layer on DBFS.

skills learned: create clusters and notebook • interact with the Databricks File System (DBFS) • use Apache Spark to read input data in CSV and JSON format • understand the characteristics and importance of the first two layers of the widely used three-layer data lake design • use Databricks functionalities

Data Manipulation

Step into the role of a data engineer working at an enterprise. Your task is to build a data lake’s serving layer and ensure that business queries run on it in a matter of seconds. You’ll start with reading cleansed data that’s already sitting in the curated layer. Then you’ll transform it, enrich it, aggregate it, and denormalize it using Apache Spark. When you’re finished, you’ll have multiple output tables that make up the serving layer of the data lake.

skills learned: perform operations and transformations on data using multiple functions offered by Spark’s SQL module • use Spark’s UDFs to enrich data and process it in a user-defined way • compare the performance of different UDF-implementation approaches • understand the importance of the serving layer

Data Manipulation and ARIMA Modeling with Pyramid

In this liveProject, you’ll investigate seasonality in hotel cancellations by building an ARIMA model that can predict cancellations on a weekly basis. You’ll learn how to manipulate a dataset with pandas in order to form a weekly time series, before going on to make your first predictions.

skills learned: dataset manipulation with pandas • time series forecasting with pmdarima

Data Pipeline with Amazon Athena

Congratulations! You’ve just been hired as a data engineer for a mobile game development studio. The company’s modern data platform architecture includes an Amazon Athena data lake and an AWS Redshift data warehouse solution. Your task is to enable batch processing of revenue transaction data by creating an end-to-end data pipeline, connecting various data sources—including user engagement events, stage controls, and public chat messaging—to the lake house solution. Using AWS CloudFormation, you’ll provision the resources required for the data pipeline. You’ll connect a MySQL data source to the AWS S3 Data Lake and transform data in the data lake using Amazon Athena. You’ll wrap up the project by creating a dynamic analytics dashboard with AWS QuickSight. When you’re done, you’ll have built a batch-processing data pipeline, start to finish, using Amazon Athena.

skills learned: create a batch ETL pipeline from RDS (MySQL) to AWS S3 • extract and process data using AWS Lambda • run SQL queries programmatically in the cloud • create data environment configurations using YAML files • test AWS Lambda locally • create an optimized ICEBERG table in Athena

Data Preprocessing

In this liveProject, you’ll design a movie recommendation system step by step, from initial development all the way to a production-ready system. You’ll begin with preparing the data, then you’ll analyze the data, and finally, you’ll preprocess and export it. Along the way, you’ll gain a firm understanding of the data and your users, which is key for developing a system that makes appropriate recommendations. While this liveProject doesn’t contain any machine learning, going through these steps will prepare you for feature engineering and hyperparameter tuning later on.

skills learned: parse and engineer features using built-in and external Python libraries • use NLP to prepare the data and design features’ vocabularies for future embeddings

Data Preprocessing

In this liveProject, you’ll test your data wrangling and data processing skills using the Julia language. You’ll step into the role of a data scientist for a real estate company with a new task from your boss—analyze and clean housing and census data for the marketing and sales teams. You’ll employ the popular Julia package DataFrame.jl as well as powerful statistics related libraries to successfully explore these datasets, and prepare them for machine learning.

skills learned: tabular data ingestion and integrity validation • exploratory data analysis using descriptive and graphical techniques • feature selection and feature engineering

Data Processing Tools with NumPy

As a data engineer in an online recruiting tech company or a large organization’s HR department, you’ll build a series of practical tools to process and extract useful information from unstructured text data using NumPy. You’ll learn important methods (including trie data structure, TF-IDF, SVD), how to implement them, and their applications in the real world. When you’re finished, you’ll have the know-how to build data processing tools that meet the needs of machine learning engineers, data analysts, and product developers.

skills learned: extract features with NumPy • vectorize text with TF-IDF and SVD • compute similar items with embedded vectors

Data Quality Check

Nextstellar Corp has recently migrated to the cloud, and for the first time, they can analyze 100% of their company’s data. But there’s a problem: your CEO isn’t confident in your data’s quality. He wants to add more data sources and collect more user behavior information, and ensure these new sources are top-notch by utilizing the Python- (or Scala-) based Deequ library. Your task is to utilize Jupyter Notebooks with AWS Glue Studio to experiment with PyDeequ for data quality assessment. Next, you’ll enhance Nextstellar’s assessment capabilities by employing AWS Glue jobs to react and take action on data quality issues. Finally, you’ll monitor data quality using CloudWatch for prompt response and maintenance of data reliability.

skills learned: Implementing Data Quality Jobs on AWS • Automating infrastructure using CloudFormation • Developing Data Quality jobs using PyDeequ and AWS Glue • Observing and monitoring Data Quality jobs.

Data Streaming in AWS

As a data engineer for a mobile game development studio, your task is to create a data streaming pipeline that collects and processes large streams of data records in real-time for lightning-fast analytics. Your company’s modern data platform architecture includes an Amazon Athena data lake and an AWS Redshift data warehouse solution. To store files, you’ll create an AWS S3 bucket, and you’ll create an AWS Kinesis delivery stream by using the boto3 library to connect to AWS Kinesis endpoints and send event data to the service. You’ll provision AWS Redshift resources and connect them to your AWS Kinesis Data Stream to analyze user behavior data to understand the user's journey inside the app. When you’re done, you’ll have a simple yet reliable data streaming pipeline that prevents resource shortages and transforms data in real-time—while it’s still relevant—ensuring more accurate data.

skills learned: create streaming data pipelines on AWS using Kinesis and Redshift • provision the resources for your data pipeline using AWS Kinesis Data Stream • use AWS Lambda to simulate an event stream • create an AWS Redshift Serverless resource using AWS CloudFormation to transform data in AWS Redshift

Data Visualization for Exploratory Analysis

Visualize this: you’re a data analyst in an online recruiting tech company or a large organization’s HR department. You’ll use Matplotlib to explore, visualize, and analyze processed data to identify missing data and outliers. You’ll build interactive plots for superior data presentation, analyze the correlation of different features using visualization methods, and create analytics dashboards for two types of users. By the end, you’ll be a better data analyst and have the skills to build storytelling tools that let you answer important business questions.

skills learned: data visualization with Matplotlib • build interactive data dashboards • solve business problems with data insights

Databases and Replication

In this liveProject, you’ll go hands on with the cross replication of an S3 bucket, and backing up DynamoDB in real time using DynamoDB streams. You’ll set up the source and target region, policies to perform replication, and then test it’s working. Finally, you’ll setup and load data using local provisioners and a solution using DynamoDB streams.

skills learned: perform cross replication with S3 • backup DynamoDB data using event driven architectures • create DynamoDB streams

Decoding Data Science Job Postings to Improve Your Resume

In this liveProject, you’ll step into the life of a budding data scientist looking for their first job in the industry. There are thousands of potential roles being advertised online, but only a few that are a good match to your skill set. Your challenge is to use data science tools to automate the process of trawling through job listings, to save time as you optimize your resume, identify the most in-demand skills, and find jobs that are a good fit for you. To do this you’ll use Python to perform Natural Language Processing and text analysis on pre-scraped data from jobs posting websites.

skills learned: process data with pandas DataFrames • K-means clustering • information retrieval with tf-dif • text cluster visualization

Deep Learning

You’ve shown your expertise in successfully modeling a time series using classical models: moving average (simple MA and exponential MA), autoregressive (AR), and autoregressive integrated moving average (ARIMA). Your next step is to determine if there’s a way to boost the performance of the time series analysis. You decide you’ll apply an unconventional approach that’s increasingly popular in finance circles: a hybrid model that combines deep learning and classical approaches. For the deep learning models, you choose recurrent neural network (RNN) and long short-term memory (LSTM). In this liveProject, you’ll focus on preparing the data for the deep learning models. The good news is that in deep learning models, you avoid a long and cumbersome preprocessing stage since, unlike in many classical approaches, they are able to detect patterns almost automatically.

skills learned: prepare data for deep learning models • run RNN and LSTM deep learning models • visualize results • evaluate model performance

Deep Learning for Multimodal Image Retrieval

Have you ever spent too long scrolling through your photos to find that one picture you know you took? In this liveProject, you’ll solve that problem—with the power of deep learning!

Your challenge is to create a working image retrieval system that can search images based on text queries: type in ‘dog’ and you’ll get pictures of dogs. You’ll combine cutting-edge natural language processing and computer vision techniques, exploit semantic word representation methods, and train a neural network on social media data using image captions as annotations. This liveProject will develop your skills with PyTorch, and be an impressive piece for your data science portfolio.

skills learned: train a Word2vec word representation method • train a CNN with PyTorch • compute similarities between words and images

Deep Neural Networks for Image Segmentation in Fine Art

In this liveProject, you’ve taken on the challenge of digitizing the collection of the World Painting Museum, and the head curator wants you to use your machine learning skills to create an index of the art. Your challenges will include classifying your training data, considering pretrained models to help aid categorization, implementing a customized CNN that can identify genre, and applying image semantic segmentation in order to facilitate the classification by identifying the elements present on the art painting.

skills learned: image transformation • build an image classifier-based convolutional neural networks • apply transfer learning for image classification and segmentation • perform image semantic segmentation to increase accuracy of art paintings identification

Deep Reinforcement Learning for Self-Driving Robots

In this liveProject, you’ll investigate reinforcement learning approaches that will allow autonomous robotic carts to navigate a warehouse floor without any bumps and crashes. Your challenges will include teaching warehouse navigation with tabular Q-learning, leveraging neural networks and Deep Q-Network models to avoid collisions, and putting together a simulation environment to handle custom reinforcement learning problems.

Please note: This project recommends using Colab Pro, which is an additional cost outside this project.

skills learned: implementing Q-learning • improving algorithm performance with deep reinforcement learning • creating a reinforcement learning environment in OpenAI Gym

Defensive Distillation

Make your model less vulnerable to exploitation with defensive distillation, an adversarial training strategy that uses a teacher (larger, more efficient) model to learn the critical features of a student (smaller, less efficient) model, then use the teacher model to improve the accuracy of the student model. In this liveProject, you’ll use a pre-trained model to train your student model without distillation, generate malicious input using FGSM, and evaluate the undefended model. Then, you’ll train a teacher model with a pre-trained dataset, train your student model with the same training set and teacher model using distillation, generate malicious input, and evaluate the defended student model, comparing the results with and without distillation.

skills learned: basics of defensive distillation • basics of logit models • CleverHans attack generator

Define and Implement KPIs

Padre Inc. uses AWS as its cloud provider and has acquired Tiddler Inc., an insurance analytics startup that uses Google Cloud Platform. As an enterprise architect at Padre, your task is to manage multi-cloud operations. Using Countly, you’ll define and implement key performance indicators (KPIs) in order to measure the success of Padre’s SaaS project that runs on AWS and Google. To ensure smooth operation of services and an optimal user experience, you’ll implement site reliability engineering (SRE) practices using Kubernetes, Prometheus, and Grafana. When you’re done, you’ll have experience using business metrics to track the health and value of your software, ensuring it provides value to customers and stakeholders.

skills learned: implement web analytics using open-source Countly

Delivery Notes Data Entry Automation with Python

In this liveProject you’ll step into the shoes of a developer at PharmaChain Iberia, a pharmaceutical supply chain company. A big global competitor has recently set up shop in your region and has been steadily gaining market share, thanks to hyper efficient business process automation in its supply lines. This automation service can receive and read delivery notes ten times faster than the admin staff in your company—and you need to innovate fast to keep your status as the regional market leader. You’ve been tasked with implementing your own company-wide automation solution, using Python to automate your processes. To achieve this, you’ll use Python to iterate through subfolders of your current text-based PDFs and PDF form files and extract field keys and values. You’ll write scripts that can identify field IDs on web forms that correspond to the PDF documents, and automatically populate the web forms with information extracted from your PDFs—no manual data entry needed! Finally, you’ll prepare a report detailing your conclusions for your manager at PharmaChain.

skills learned: automate repetitive manual processes in Python • extract data from PDFs • web page resource identification • dynamically assign values to web form fields using autogenerated JavaScript

Demographic Data Analysis

In this liveProject, you’ll use the Python data ecosystem to explore how demographic data affects American life expectancy. You’ll use pandas and NumPy to clean and merge a newly collected data set Matplotlib, and seaborn to create a variety of plots that showcase the distribution of the data and expose the relationships between the variables.

skills learned: reading, cleaning, and merging data with pandas • exploring data for trends and distributions • doing data visualizations with Matplotlib and Seaborn

Deploy a DL Model

In this liveProject, you’ll deploy a completed deep learning model so that it can be easily used by your clients. You’ll use the Flask Python library to build a web page that will host your model, and use HTML, CSS, and JavaScript to build a simple web interface. Finally, you’ll take your model out into the world by using ngrok to make locally-served web pages available anywhere.

skills learned: create a web server with Flask • build a simple interface with HTML and CSS • make locally served web pages available to users on other systems

Deploy a Fargate Cluster

You’re the lead DevOps engineer in an IT department with about 60 engineers. Your director wants to transition the company from a traditional cloud server to containerization while minimizing operational workload. Impressed with AWS Fargate’s ability to handle containers without managing servers, he’s tasked you with deploying a pilot project using Fargate. You’ll create a proof-of-concept by running an NGINX image on AWS Fargate. To ensure that the infrastructure meets the spec of being reusable as code, you’ll use AWS CDK along with Python as your programming language. When you’re finished, you’ll have investigated, evaluated, and deployed an NGINX container along with all the required resources into AWS Fargate.

skills learned: set up basic network infrastructure with AWS CDK • create a cluster with AWS CDK and other required resources (e.g. Load Balancer, Execution Role) • write a task definition for creating Fargate tasks and Fargate services

Deploy a Flask Application

Your assignment as lead DevOps engineer is to deploy a REST API Flask application into AWS Fargate. You already have the application code and the docker-compose file to run the API locally. You’ll package the code in CDK, build the images, store them in Elastic Container Registry (ECR), then use them for deployment in Fargate. When you’re finished, you’ll have practical experience deploying a Flask application, as well as a solid understanding of some of the services Fargate requires for deployment.

skills learned: use ECR to host private Docker image • deploy an existing Dockerized service into AWS Fargate Cluster • create the Task Definition file

Deploy a Forecasting Model

In this liveProject, you’ll architect a solution to serve predictions from time series forecasting models over a REST API. Once you’ve architected your solution from a high-level perspective, you’ll monitor and assess the performance of the model and potentially undertake retraining to improve accuracy.

skills learned: serve our predictions with a REST API using FastAPI • validate the API query parameters by using Python’s type annotations and pydantic • track our model’s performance with MLflow

Deploy a Predictive Model

In this liveProject, you’ll deploy a machine learning model to production so it can be easily used by colleagues. You’ll create a virtual environment for deploying your application, use Flask to build a local web service that returns predictions, and Heroku for remote deployment.

skills learned: create a virtual environment • GET and POST requests • create a web framework for a prediction app • remotely deploy an app on Heroku

Deploy a Predictor on Android

In this liveProject, you’ll create an Android application that can run a pretrained basketball predictor deep learning model for the easy use of your client. Your challenges will include converting the DataFrames into JavaScript arrays, converting your model into a TensorFlow Lite model, and finally packaging the model inside a working Android application.

skills learned: Converting a model to Android use • generating JavaScript usable data sets • simple Android development

Deploy a Predictor on the Web

In this liveProject, you’ll deploy a pretrained basketball predictor deep learning model onto the web for easy use by clients. You’ll utilize the powerful TensorFlow.js framework to ensure the model works in the browser, as well as converting DataFrames into JavaScript arrays, and building a simple website around the model and data sets.

skills learned: converting a model to web use • generating JavaScript usable data sets • simple web development

Deploy a Streamlit Dashboard

In this liveProject, you’ll build an interactive dashboard that will allow the marketing team at your company to monitor any mention of your company’s products on Reddit. You’ll start by visualizing relevant subreddit data, then build a model to monitor your mentions on Reddit. Finally, you’ll deploy the Streamlit dashboard on Heroku. Streamlit offers a simple and easy way to build a highly interactive and beautiful dashboard with just a few lines of codes, whilst Heroku offers free web hosting for data apps.

skills learned: dashboard design • data visualization • integrating API call with a dashboard

Deploying Models

In this liveProject, you’ll take on the role of an engineer at AISoft, where you'll be part of two dynamic teams: MLOps and Core-ML. On the MLOps team, you'll utilize software engineering techniques to ensure your models are not only accurate but also scalable, reliable, and maintainable. You’ve been assigned two important support tasks for the Core-ML team.First, you'll utilize the Gradio Python library to create an interactive web user interface that runs MRI classification and segmentation transformer models in the background, and then you'll build a pipeline that provides interpretability to the decisions of a construction vehicle classification model.

skills learned: build a web interface for MRI classification and segmentation using Gradio • run a vision transformer model for MRI classification and a Segformer or Maskformer model for MRI segmentation via the interface • implement Grad-Cam and LIME to explain transformer predictions

Deploying a Deep Learning Model on Web and Mobile Applications

In this liveProject, you’ll take on the role of a data engineer working for an app development company. Your boss has a great idea for a new app that can automatically identify what food is shown in an image—perfect for when you want to post your lunch on social media! They have asked you to build it for them.

Your challenge is to train an image classification system using TensorFlow and then deploy that system to production so that it can run in the web browser and as a mobile application. To do this, you’ll make use of the Python and TensorFlow ecosystems to create your model, then package it for deployment using Docker and Expo.

skills learned: train a deep learning model • deploy a deep learning model using TensorFlow.js • create web applications • deploy your application using Docker

Detect Eyes Using Haar Cascades and Hough Transformation

In this liveProject, you’ll upgrade facial recognition security to more complex iris recognition. You will read an image, draw bounding boxes around the detected eyes and detect the circular eye area region, then save the output image.

skills learned: reading and writing images • using Haar Cascade Classifier for eye detection • using Hough Circle Transform for circle detection

Detect Faces in Videos

Face detection has numerous applications across software and is an essential part of the pipeline for detecting deepfake videos. In this liveProject, you’ll implement a component that detects faces in videos, and normalizes them to the same size and visual appearance. This component is a vital foundation for a deepfake detection system.

skills learned: detect and crop faces • normalize and scale faces • align faces according to the eyes locations

Detect Sentiment with Transformers

Finative, the environmental, social, and governance (ESG) analytics company you work for, analyzes a high volume of data using advanced natural language processing (NLP) techniques to provide its clients with valuable insights about their sustainability. Your CEO has concerns that some of the companies Finative analyzes may be greenwashing: spreading disinformation about their sustainability in order to appear more environmentally conscious than they actually are.

As a data scientist for Finative, your task is to validate your sustainability reports by creating and analyzing them. You’ll compute conditional probability with Bayes’ Theorem, by hand, to better understand your model’s performance through metrics such as recall and precision. You’ll learn an efficient way to prepare your data from different sources and merge it into one dataset, which you’ll use to prepare tweets. To successfully classify the tweets, you’ll use a pre-trained large language model and fine-tune it using the Hugging Face ecosystem as well as hyperopt and Ray Tune. You’ll use TensorBoard and Weights & Biases to analyze and track your experiments, and you’ll analyze the tweets to determine whether enough negative sentiment exists to indicate that the company you analyzed has been greenwashing its data.

skills learned: conditional probability • fine-tuning a large language model • hyperparameter optimization • monitor training experiments

Detect a Face Using Haar Cascades

Welcome to the era of digital security! In this quick liveProject, you’ll develop an efficient solution for detecting faces in a live video stream. You’ll need to read an input video from a webcam, detect faces present in the input video stream, draw bounding boxes around the detected faces, and then write the detected faces to a new video.

skills learned: reading and writing videos • using a Haar Cascade Classifier for face detection

Detecting Unusual Data

In this liveProject, you’ll build your own Python data library that can detect unusual data. This data can affect regression analysis, and so your library will help spot it before it can negatively affect results. You’ll program functions to detect high-leverage points, outliers, and influential observations. Through the analysis of multiple examples, you'll gain a deep understanding of the kind of problems that can arise with real data.

skills learned: detecting unusual observations • visualizing and analyzing data with Matplotlib, seaborn, and statsmodels

Detection with SVM

In this liveProject, you’ll develop a machine learning solution that can identify the difference between faces in deepfake videos and real faces. You’ll use a support-vector machine (SVM) classifier approach to determine which videos have the artifacts associated with deepfakes, and combine face detection, feature extraction, and your SVM classifier together in one pipeline to create your detection system.

skills learned: Training an SVM classifier resulting in deepfake detection system • Evaluating a system on the test set

Deutsch-Jozsa Algorithm

In this liveProject, you’ll create and utilize quantum oracles to determine whether a given function is balanced or constant. You’ll utilize unique quantum superposition to determine algorithmic accuracy over just a single pass of the outcome, potentially massively speeding up systems.

skills learned: write a quantum algorithm with Amazon Braket SDK • implement a quantum circuit to determine whether a given function is balanced or constant • use oracles with a quantum circuit • implement input and output qubit registers

Develop Quantum Algorithms

As the researcher for venture capital firm Action at a Distance Corporation your job is to evaluate proposals from quantum startups hoping to secure an investment. But first, you need to expand your knowledge of quantum computing.

You’ll build your quantum muscles by investigating and coding key quantum algorithms including the quantum teleportation algorithm, the quantum phase estimation algorithm, the quantum Fourier transform algorithm, and Grover’s algorithm. With your new knowledge, you’ll compose a brief report about whether or not now is the right time to invest in quantum startups.

skills learned: Shor’s algorithm concepts/techniques • quantum superdense coding teleportation • quantum Fourier transforms • quantum phase estimation • quantum oracles

Develop Z-score and Baseline Results

Failure is not an option for Sigma Corp. As a lead data scientist for the large conglomerate of energy production companies, it’s up to you to help ensure interruption-free operations by developing a means for detecting anomalies that signal potential problems. Using metrics, including the receiver operating characteristic (ROC) curve and the area under curve (AUC) score, you’ll evaluate anomaly detection algorithms. You’ll build a z-score anomaly detection algorithm, which focuses on a single feature and provides a simple benchmark, and you’ll apply it to a dataset to establish a reference for comparison. When you’re finished, you’ll have a firm grasp of z-score anomaly detection, classification error categories, and evaluating anomaly detection algorithms.

skills learned: z-score method • evaluate algorithms using ROC AUC metrics • basic synthetic anomalies

Dictionaries

In this liveProject, you’ll utilize Python dictionaries to efficiently combine a file of product info and a file of sales entries. You’ll take on the role of a data engineer in a company that has been modernizing its sales systems, and tackle a common data merging task to help generate useful reports. Solve the challenges of this project, and you’ll understand the real power and flexibility of dictionaries and how they are used literally everywhere in Python.

skills learned: basic dictionary manipulation for adding and accessing data, in, get() • ways to handle missing keys

Difference in Differences

In this liveProject, you’ll use difference-in-differences as an alternative to A/B testing to assess the discounting strategy of an e-commerce company. This approach compares growth between a group that has been experimented on, and data from a control group. You’ll use Python and causal inference to examine the growth trajectory in profits from customers in a high discounts group against customers that didn’t get a discount.

skills learned: when and how to use difference-in-differences • estimate the impact of discount on profits using diff-in-diff

Dimensionality Reduction

In this liveProject, you’ll build a movie recommendation system that reduces the dimensionality of the spaces involved to enhance its performance so that latent representations are identified, which aim to capture users and items interactions with the system on the same latent vector space. You’ll work to compute clusters on the movies and clients of your site, and utilize latent representations of users and items with a reduced dimensionality to compute similarities. Once you’ve established this data, you’ll build and test your recommendation engine.

skills learned: compute clusters of users and items based on similarities • compute similarities on top of learned latent representations based on user and item interactions • define recommender systems based on similarity clusters

Dimensionality Reduction with PCA, t-SNE and UMAP

In this liveProject, you’ll use the Julia programming language and dimensionality reduction techniques to visualize housing sales data on a scatter plot. This visualization will allow the marketing team to identify links and demand patterns in sales, and is also a useful tool for noise reduction or variance analysis. You’ll use the popular PCA algorithm to visualize the sales dataset with overlaid clustering assignments from k-means and DBSCAN methods, and then expand Julia’s capabilities by calling Python modules using the PyCall.jl package. This extra flexibility will allow you to explore the t-SNE and UMAP algorithms which have excellent results for high-dimensional datasets.

skills learned: PCA, t-SNE, and UMAP dimensionality reduction techniques • validating and analyzing output of PCA algorithm • calling Python modules from Julia

Discovering Disease Outbreaks from News Headlines

In this liveProject, you’ll take on the role of a data scientist at the World Health Organization (WHO). The WHO is responsible for responding to international epidemics, a critical component of which involves monitoring global news headlines for signs of disease outbreaks. However, this daily deluge of news data is too huge to manually analyze. Your challenge is to pull geographic information from headlines, and determine where in the world outbreaks are occurring. Problems you will have to solve include extracting information from text using regular expressions, using the Basemap Matplotlib extension to visualize map locations for patterns indicating an epidemic, and reporting your findings to your superiors so resources can be dispatched.

skills learned: text extraction with pandas • clustering with K-means and DBSCAN • visualize clusters

Double Machine Learning Approach

In this liveProject, you’ll utilize machine learning for treatment effect estimation in order to help an e-commerce company deliver targeted discounting to the most profitable customers. You’ll build a model that predicts the effect of a discount on a customer following a causal model, maximizing profits for the business.

skills learned: pandas to manipulate tabular data • linear regression for causal inference • use machine learning to make causal models for personalization

ECOD Algorithm

Sigma Corp, a large conglomerate of energy production companies, has recently implemented anomaly detection algorithms and is generally pleased with their performance. However, analysts report that not all anomalies are being identified and the algorithms are too slow at times. As a lead data scientist at Sigma, it’s up to you to address these concerns. To increase the robustness of the algorithms, you’ll implement and optimize the probability-based Empirical Cumulative distribution-based Outlier Detection (ECOD) method, an alternative to statistical methods. You’ll benchmark the ECOD method in order to compare its performance with the statistical MD and PCA methods Sigma is currently using. When you’re finished, you’ll have firsthand experience implementing the highly efficient ECOD method to detect anomalies in multidimensional data.

skills learned: ECOD method

Ensure Computational Governance

The development teams at Messflix, a movie and TV-show streaming platform are pushing domain data products through their newly implemented data mesh. Now the CTO has tasked you, their consultant, with striking a balance between the benefits of the data mesh, the freedoms the data products have, their usefulness, their interoperability, and aspects of their security. You’ll use Python and pandas to write policies that check the registration and pushed data, helping users provide all the required registration information. You’ll create tooling to classify data into categories for improved data labeling, and protect sensitive data with pseudonymization functions. When you’re done, you’ll have learned skills for federated computational governance that balance the benefits of data products with the benefits of the data mesh.

skills learned: understand computational governance • implement data policies for data products as well as individual data pieces

Escape Room

In this liveProject, you’ll use Python to build a simple text-based escape room game to help enhance the experience of a real-world escape game. This project involves following all the steps in a software development process, from reviewing your client’s given requirements, to designing, building, and testing your code. You’ll gain experience operating within a set of requirements and refactoring your code to optimize class definitions, tighten up functions, and make sure it’s Pythonic!

skills learned: software design • assertions • exceptions • error handling • refactoring • modularizing code • software quality • fulfilling software requirements

Estimating the Profitability of Discounts

In this liveProject, you’ll utilize causal inference techniques to help an e-commerce company estimate the impact of discounts on profits. You’ll learn how bias gets in our way of inferring the effect of discounts, and how to adjust for it for more accurate results. Leverage adjustment techniques, like linear regression, to adjust for bias and see how linear regression can improve the quality of your data. Your final task is to present a recommendation on whether the company should distribute its discounts or not.

skills learned: how bias gets in the way of causal inference • adjust for bias using a linear regression

Evaluate a Binary Classifier

Judging the effectiveness of a machine learning model requires in-depth analysis. This quick liveProject builds on the work you have completed in Machine Learning for Classification. You’ll assess your early models and consider better alternatives. You’ll plot the ROC curves of the model and compare it to multiple dummy models, and tune your hyperparameters to deliver the most accurate results possible.

skills learned: evaluate predictions and accuracy • visualize patterns with classes • conduct hyperparameter tuning • prepare a model for production deployment

Evaluate and Explain DL Models

In this liveProject, you’ll implement model performance metrics that can test the effectiveness of your models. You’ll calculate accuracy, precision, F1 score and recall values from the classification results for an existing model, and then estimate the ROC curve and AUC value. Finally, you’ll create a Gradient Class Activation Map. This map can highlight features and regions in an image that the deep learning model finds important, and manually inspect whether the model is performing in the desired way.

skills learned: make predictions using deep learning models using Keras • implement Grad-CAM visualization • implement deep learning model performance metrics • create a radar plot to display the performance of multiple models

Exceptions

In this liveProject, you’ll explore the unique way that exceptions work in Python. Python makes frequent use of exceptions, but they often function in a very different way than other programming languages. You’ll experiment with the basic try-and-except structure used for handling exceptions, then dive into using inheritance with exceptions, and creating your own hierarchy of exceptions.

skills learned: how Python exceptions are created, raised, and handled • what is in an exception object • how inheritance works for exceptions

Expert Systems

In this liveProject, you'll take over administering a zoo automatically by crafting three different kinds of expert systems to help customers determine what animal they’re looking at. An expert system captures an expert's know-how by encoding it in a knowledge base. You’ll create a prototype 20-Questions-style game expert system, then expand it with logical rules and Boolean statements to build more powerful and realistic expert systems.

skills learned: expert system concepts • the Animal Game

Explore IMDb Dataset

In this liveProject, you’ll explore a dataset of movie reviews and prepare it for sentiment analysis. The dataset you've provided is balanced between positive and negative reviews, but is encoded in such a way that you will need to use the dictionary (lookup) in this dataset package to decode the content to plain text. Your challenges will include decoding the data to plain text, and then converting the plain text into tokens in a Pythonic manner.

skills learned: use a dictionary to decode and encode data • learn NumPy array indexing techniques • understand Python dictionary data structure, and how to edit and manipulate it

Fake News Detection

In this liveProject, you’ll use the RoBERTa variation of the BERT Transformer to detect occurrences of fake news in a data set. Political news can be tricky to validate for accuracy, as sources report the same events from different biased angles. RoBERTa uses different pre-training methods than traditional BERT and has hyperparameters that are highly optimized, meaning it tends to perform better than its predecessor. You’ll start out by loading the model using the Hugging Face library and training it to your data with PyTorch Lightning. The project will also include the implementation of training a custom tokenizer from scratch and using it to tokenize the data. A successful model will maximize positives, and so model evaluation will be based on your model having a high recall score.

skills learned: loading and configuring pretrained RoBERTa model using Hugging Face • building and training a text classifier using PyTorch Lightning • validating the performance of the model and quantifying the results

Fast.ai

In this liveProject, you’ll create a recommendation engine for an online store using the Fast.ai library. You’ll utilize a dot product and a neural network to come up with the latent factors in a rating matrix, and compare and contrast them to determine which is likely to deliver the best recommendations. You’ll need to select and clean your data, pick the right methods, then create the functions that you need to recommend products based on predicted ratings.

skills learned: selecting, cleaning and choosing data for collaborative filtering • matrix collaborative filtering techniques using Fast.ai collab_learner class • learning latent factors for collaborative filtering recommenders

Feedback-Loop and Exploration Methods

Since recommender systems train and learn over the data they recommended themselves, they will never train over, learn, or recommend items that they didn’t already recommend for some reason, such as insufficient ranking to be seen by the user. It’s important to break this Feedback Loop in order to ensure that suitable recommendations aren’t missed. But you must strike a balance between deviating (just enough) from the system’s predictions through exploration and not defeating the system’s purpose altogether. In this liveProject, you’ll learn three methods of exploration that help you provide better recommendations to your users, as well as the costs and benefits of each.

skills learned: rerank a list of items after a model already predicted their rankings • add noise to the model to affect predictions as they’re made • combine lists with different rankings to increase traffic to lower ranking items • use a Feature Store as part of a prediction pipeline

Find Hidden Connections with Graphs

You’ve set your sights on a collection of prime off-the-market New York City properties whose value is likely to double in the next couple of years. With your excellent sales skills, you’re confident you can get the owners to sell at a fair price, if you can determine who the owners are. You have a knowledge graph—built from tax records, property deeds, and permits—that identifies entities associated with the properties. These entities might not be the true owners, but identifying them could help you determine who the true owners are.

Your task is to transform the knowledge graph into a representation that can be processed by a machine learning model later. Using simple graph and NLP techniques, including node2vec—one of the most influential algorithms in the graph community—you’ll improve the quality of the data by removing the noise from the knowledge graph. You’ll convert the nodes in your graph into embeddings, analyze how well your embeddings represent the underlying knowledge graph, and develop insights on tuning the node2vec hyperparameters. When you’re done, you’ll have learned techniques for analyzing and visualizing embeddings and associating them to the original graph, helping you determine who the “hidden” owners are.

skills learned: intermediate Neo4j (Graph Data Science library) • intermediate graph theory (weakly connected components) • build a node2vec model with Neo4j

Fine-Tune the Llama Model

Your local school district has an LLM chatbot specialized in chemistry. But it sometimes answers questions entirely unrelated to chemistry! To improve the safety of this LLM application, you are asked to come up with a new LLM that would classify student questions and return machine-readable (JSON) messages to the application frontend so that the application can decide whether to answer this question. To accomplish that, you need fine-tuning to "teach" the model how to classify questions through many examples, which themselves are generated by a Llama-based LLM, and to always respond with JSON.

skills learned: Develop data set of finetuning • Finetune an LLM

Forecast using Deep Learning

In this liveProject, you’ll use deep learning to implement powerful time series forecasting models that can beat the performances of previous models. You’ll work with the Python package “PyTorch Forecasting” and the deep learning models LSTM and N-BEATS. You’ll also get experience with key techniques of cross learning, ensembling, and hyperparameter tuning.

skills learned: perform forecasting with LSTM and N-BEATS models using PyTorch Forecasting and PyTorch Lightning • optimize models with Bayesian optimization using Optuna • implement callbacks for customized model training

Fractals

In this liveProject, you’ll use recursive and non-recursive algorithms to generate fractals for graphic design. Fractals are shapes that have fractional dimensions and can produce very beautiful and interesting visualizations. You’ll start by exploring recursive line-drawing fractals, move on to strange attractors, and finish up with escape-time fractals.

skills learned: using tkinter • drawing in the Canvas widget • using mouse events to let the user select part of an image

From Data Lake to Lakehouse

Turn an existing data lake into a Lakehouse, using Delta Lake, an open table format (and the cornerstone of Databricks’ Lakehouse design). For data processing and interacting with the Lakehouse, you’ll use Apache Spark. As you transform the existing tables into Delta tables, you’ll explore Delta Lake’s rich features, see firsthand how it handles potential problems, and appreciate the sophistication of the Lakehouse design.

skills learned: leverage Delta Lake capabilities including schema enforcement and time travel • use Delta Lake commands • interact with multiple layers of the three-layer architecture • use Apache Spark for data processing and interacting with the Lakehouse

Functions as Objects

In this liveProject, you’ll create Python functions for specific data cleaning operations to help import product data from a legacy system into a new pipeline. You’ll work with a delimited text file with unhandy export conventions, and utilize functions to clean and convert fields for cleaner and seamless data transfer. You’ll even break your new operations down into separate functions so that you can easily reuse your code at a later date.

skills learned: functions as dictionary values • mutable objects as default function parameters • how to use and create decorators

GNN for Document Classification

In this liveProject, you’ll implement a Graph Neural Network (GNN). This powerful model will allow you to use the document content from the first liveProject combined with the structure of the citation graph from the second liveProject to build an even more powerful model—one that will predict the sub-field of statistics of each of your customers’ papers.

skills learned: node classification by text embedding signals and citation graph structure signals • build a GNN classifier with PyTorch Geometric

Generate Art

In this liveProject, you’ll replicate Google’s Deep Dream algorithm to explore the artistic creations of a neural network. You’ll start by investigating the latent space of a deep neural network and how manipulating its values can affect an input image. You’ll visualize inputs that maximize your filters, and manipulate these filters to produce ‘dream-like’ hallucinations.

skills learned: mathematical operations on images • implementing loss functions • transfer learning

Generate Text Samples

In this liveProject, you’ll build a deep learning model that can generate text in order to create synthetic training data. You’ll establish a data training set of positive movie reviews, and then create a model that can generate text based on the data. This approach is the basis of data augmentation.

skills learned: building and training a deep learning model for text classification • using sklearn module to report model classification metrics • condition based sampling technique for NumPy array

Generative Modeling with VAEs and GANs

In this liveProject, you’ll utilize unlabeled data and unsupervised machine learning techniques to build and train data generative models. You’ll employ generative modeling, such as a variational autoencoder (VAE) that can generate new images by sampling from the latent distribution. You’ll then use an unsupervised generative adversarial network to generate new images.

skills learned: implement autoencoders to reconstruct images • utilize variational autoencoders for image generation • train an unsupervised deep convolutional GAN

Get Started

Start building your robotics foundation: You’re a programmer helping create prototype robots for the RoboPrototypes company. You’ll get a brief introduction to ROS 2, set up the programming environment, and onboard robotics building blocks with basic tutorials. Then you’re ready to create launch files, which are integral to every ROS project. As you create packages in ROS 2, you’ll learn to define and use parameters and declare custom messages. When you’re finished, you’ll have a firm foundation for delving deeper into robotics.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: set up the programming environment for ROS 2 • ROS 2 fundamentals • create a ROS 2 workspace • create ROS 2 launchfiles, services, messages

Getting Started with Google Colab Using PyTorch

In this liveProject, you’ll get hands-on experience using the powerful Google Colab tool for machine learning and deep learning. Colab notebooks let you execute your data science code in Google’s cloud, getting all the benefits of Google’s incredible hardware. You’ll see how Colab works for yourself by running through simple machine learning tasks such as data preprocessing, making use of Colab’s free GPU and TPU hardware acceleration capabilities, and combining Colab with scikit-learn and PyTorch to train a classifier.

skills learned: using Colab as a Jupyter Notebook • utilizing Colab hardware acceleration capabilities • combining Colab with PyTorch

Getting Started with Google Colab Using TensorFlow

In this liveProject, you’ll get hands-on experience using the powerful Google Colab tool for machine learning and deep learning. Colab notebooks let you execute your data science code in Google’s cloud, getting all the benefits of Google’s incredible hardware. You’ll see how Colab works for yourself by running through simple machine learning tasks such as data preprocessing, making use of Colab’s free GPU and TPU hardware acceleration capabilities, and combining Colab with scikit-learn and TensorFlow to train a classifier.

skills learned: using Colab as a Jupyter Notebook • utilizing Colab hardware acceleration capabilities • combining Colab with TensorFlow

Getting Started with Jupyter Notebook

In this liveProject, you’ll get hands-on experience using Jupyter Notebook in a real-world data science project. You’ll train a simple KNN classifier and use Jupyter, IPython, and the easy-to-use Markdown markup language to document and share your work. Your challenges will include customizing your notebooks, incorporating your notebooks into a data science project, and sharing your projects with the community on GitHub.

skills learned: collaborate on Jupyter Notebook projects • customize your notebooks • work with Jupyter Notebook extensions

Graph Embeddings for Document Similarity

Now that one of the data scientists at your publishing company was able to extract citations from the full text of the representative text corpus, your task is to generate new vector embeddings based on the citation graph, then cluster the documents using these embeddings to gain insight from the graph structure.

skills learned: converting graph nodes to vectors • dimensionality-reduction • clustering

Graph Structure in Text Data

Imagine you work for a company that publishes scientific articles created by its customers—primarily researchers and scientists in the field of statistics—whose volume of research has grown so large that it’s not possible for your company to read every paper nor for the researchers to stay on top of everything happening in the field. Your task is to use automated means to find some structure in the collection of scientific articles so that the company can more easily home in on the researchers’ interests and help them research more effectively. To do that, you’ll generate document embeddings, use them to impute a document graph, and execute graph algorithms against this graph in order to generate insights from it.

skills learned: convert text into graphs • explore centrality algorithms for clustering using Neo4j

Graphs from Real-World Data

New York City real estate is a trillion-dollar market, and you’ve got your eye on a collection of prime NYC properties whose value will likely double in a couple of years. They’re not currently for sale, but with your excellent sales skills, you’re confident you can get the owners to sell at a fair price, if you can only determine who the owners are. To obtain the owners’ contact data, you’ll construct a knowledge graph from publicly available data that contains tax records, property deeds, and permits. You’ll scan the data for the target owners, analyze the datasets for possible relationships, develop a knowledge schema that can extract insights into your use case, and load the data into Neo4j to query and visualize the connections. When you’re done, you’ll have practical experience applying widely used graph tools to real-world data, and you’ll understand how different choices for your graph schema can lead to different insights.

skills learned: intermediate Neo4j (graph query using the Cypher language, loading and viewing graphs via Neo4j Desktop)

Grover's Search to Recover an ISBN

In this liveProject, you’ll use the powerful Quantum SDK from Microsoft to develop a simple quantum application using Grover's search algorithm. This algorithm will be able to generate missing data, such as by taking a query in the form of an ISBN with one or several missing digits, and recovering those digits to return the information about the book with this ISBN. You’ll start simple by implementing the algorithm to recover only a single missing digit, then expand to recovering several missing digits. Finally, you’ll set up a classical query of an online ISBN database to find relevant book information.

skills learned: designing and writing simple Q# code • integrating quantum code with classical pre- and post-processing routines

Grover's Search to Solve a Cryptarithm

In this liveProject, you’ll use quantum computing to solve a cryptarithm—a mathematical equation in which digits are replaced by alphabet letters. Your challenges will include figuring out how to represent the problem in a way suitable for Grover’s search algorithm, then implementing the algorithm and writing unit tests to validate your solution.

skills learned: formulating problems in terms of quantum oracles • designing and writing Q# code based on algorithm descriptions • integrating quantum code with classical pre- and post-processing routines

GuardDuty Intrusion Detection

In this liveProject, you’ll set up the intrusion detection service Amazon GuardDuty to help catch any hackers who may be trying to break in—or who already have and are up to no good! To achieve this goal, you’ll iteratively develop infrastructure as code in the form of AWS CloudFormation templates, then learn to address cross-account automation issues.

skills learned: develop automation to configure Amazon GuardDuty across accounts • develop Python code to retrieve a threat intelligence set from AlienVault OTX • deploy to a scheduled AWS Lambda Function to keep the threat intelligence set up to date • configure Amazon GuardDuty to use the intelligence set

GuardDuty Test and CI/CD

In this liveProject, you’ll ensure that the security team can make incremental, verifiable changes to its Amazon GuardDuty intrusion detection configuration. You’ll develop a suite of automated tests and CI/CD AWS CodeBuild pipelines to deploy and test changes across the organization.

skills learned: develop automated tests to verify the configuration of the Amazon GuardDuty service and AWS Lambda function • develop a CodeBuild pipeline that will trigger on changes to the code of the AWS Lambda Function • develop a CodeBuild pipeline that will trigger on changes to the infrastructure as code

Handling Sensitive Data

In this liveProject, you’ll develop a machine learning model that can make a personalized prediction of whether an individual has COVID-19. You’ll work with a fictional audio dataset of digital footprints from smart phones, applying logistic regression and decision trees to turn this dataset into a COVID-19 predictor. You’ll learn how to formulate a ML problem by identifying data points, and their features and labels, so that you can take advantage of ready-made ML methods.

skills learned: basic flow control in Python • create and manipulate figures in Python • create and manipulate NumPy arrays to represent matrices and vectors

Hash Tables and Unit Tests

Your real estate developer client has a large sorted dataset and the ability to zoom in on select parts of the data, thanks to the programs you’ve written for them previously. Now they’ve noticed duplicate entries in the dataset, and they’ve asked you to solve this problem. In this liveProject, you’ll create a program that will read in the large CSV data file, leveraging a hash map (which you’ll test and implement) in order to discard duplicate entries as quickly as possible. Though you’ll use the csv and unittest libraries in Python, you’ll write your own unit tests—a fundamental software engineering skill and a practical requirement on most real-world projects. Finally you’ll test your tests against Python’s existing dictionaries.

skills learned: write code manipulating Python dictionaries • write unit tests involving dictionaries • cover a variety of testing scenarios • implement a basic hashmap from scratch • use your hashmap to eliminate duplicate entries

Hate Speech Detection

In this liveProject, you’ll use the ALBERT variation of the BERT Transformer to detect occurrences of hate speech in a data set. The ALBERT model uses fewer parameters than BERT, making it more suitable to the unstructured and slang-heavy text of social media. You’ll load this powerful pretrained model using the Hugging Face library and fine-tune it for your specific needs with PyTorch Lightning. As falsely tagging hate speech can be a big problem, the success of your model will involve calculating and optimizing its precision score. Your final product will run as a notebook on a GPU in the Google Colab environment.

skills learned: loading and configuring pretrained ALBERT model using HuggingFace • building and training a text classifier using PyTorch Lightning • validating the performance of the model and quantifying the results

How to Think about Automating Email

In this quick liveProject, you’ll automate the process of filling out a custom email from a CSV of email addresses and exam scores. You’ll then implement a randomizer to assign a random discussion partner, all using a Python library of your choice.

skills learned: using the smtplib library • reading documentation for Python • parsing a CSV file to store information in a data structure

How to Think about Manipulating Data

In this quick liveProject, you’ll use the pandas library to read a table from a Wikipedia page into a DataFrame, clean and arrange the data into columns, and extract useful information quickly to reveal data insights.

skills learned: organizing and visualizing data in a pandas DataFrame • reading, working, and experimenting with new Python libraries

How to Think about Scheduling Tasks

In this quick liveProject, you’ll develop a simple task scheduler app. Your challenges will include coding functionality that will arrange a day’s tasks to either minimize time left over during a day, or to provide the greatest variety of tasks during a day.

skills learned: implementing an optimization algorithm • recursion method to call a function • writing code to fit certain constraints • using brute force to find a solution

How to Think about Simulating Nature

In this quick liveProject, you’ll write a UI that simulates water drops on a window during rain. You’ll use nested lists, the random library, and tkinter to increase droplet size with certain probability, and resetting sizes when drops become too big.

skills learned: using the tkinter library • adding widgets to a UI • manipulating shapes in a UI • working with row and column data in a grid

How to Think about Solving a Maze

In this quick liveProject, you’ll design and implement an algorithm to solve a simple maze, revealing a path to the exit.

skills learned: implementing a wall-following algorithm • testing and debugging code • appropriate data structures • understanding algorithms, including edge cases

Human Pose Estimation with Deep Neural Networks

In this liveProject, you’ll take on the role of a machine learning engineer working for a company developing augmented reality apps. These apps include games, virtual shopping assistants, and fitness coaches that need to be able to reliably recognize the shape of a human body. Your challenge is to create an application for human pose estimation: detecting a human body in an image and estimating its key points such as knees and elbows. To do this, you’ll build a convolutional neural network from scratch, training your model using Google Colab and your GPU. At the end of this liveProject, you’ll have completed an interactive demo application that uses a simple webcam to detect and predict human keypoints.

skills learned: object detection and keypoint detection algorithms • transfer learning • training CNNs on images • deploy human pose estimator using webcam

Hybrid Model

Now that you’ve prepared the data for the deep learning models, you’re ready to apply the hybrid model, which leverages the strengths of both the classical and deep learning models. In this liveProject, you’ll focus on the essential steps for running a time series analysis. You’ll start with ensuring the data you prepared is ready for processing with the deep learning models. Next, you’ll run the RNN and LSTM for the separate datasets. Finally, you’ll evaluate the performance of your models.

skills learned: combine classical and deep learning models • visualize results • evaluate model performance

Image Classification

In this liveProject, you’ll join BrainAI’s MRI data analysis team. BrainAI needs you to develop a state-of-the-art AI module utilizing vision transformer models to detect brain tumors with exceptional accuracy. Armed with Python tools like Hugging Face Transformers, PyTorch, and more, you'll detect the presence or absence of tumors within human-brain MRI datasets. With Google Colab's GPU computing resources, you'll utilize deep learning to try and achieve a 95%+ accuracy rate.

skills learned: set up training, test, and validation datasets with biomedical (MRI) images • train and evaluate a vision transformer model for brain tumor classification

Image Classification with Deep Learning

In this liveProject, you’ll step into the role of a developer working for car rental company CarsUnlimited. You have a bold new idea for an app that can revolutionize your business. You envisage a service where a customer can snap a picture of any car on the street, and instantly find out if CarsUnlimited has that make and model available. Your colleagues have the app UI covered; your challenge is to build the algorithm to classify the cars that were snapped by the user. You decide to develop a machine learning algorithm for solving this problem.

You’ll start with cleaning your data, before experimenting with classical machine learning algorithms such as SVMs, Random Forests, Decision Trees, and assess their performance metrics. With an entire range of parameters to tune and improve, you will learn about how to automate this search. From there, you’ll dive into building a deep neural network and explore different search strategies to finalize your powerful new neural network architecture.

skills learned: identify when to use classical machine learning algorithms or deep learning for classification • build a neural network architecture and automate the search for parameters

Image Processing

In this liveProject, you’ll use the Python Imaging Library (PIL) to tackle common image processing tasks. You’ll learn about some of the algorithms that let you apply filters to images, and you’ll use PIL to adjust image brightness, contrast, and color tone. You’ll crop, rotate, reflect, and perform other geometric operations on images.

skills learned: loading an image file into a PIL image • displaying a tkinter PhotoImage • making tkinter dialogs • applying various PIL image transformations to enhance and modify images

Image Segmentation

In this liveProject, you'll pioneer the development of cutting-edge MRI segmentation algorithms using transformer architecture for computer vision company VisionSys. Manual segmentation is labor-intensive and expensive, so you’ll be developing a custom model that can do it for you. You'll train and evaluate SegFormer and MaskFormer models to identify brain tumor regions with over 90% accuracy. With Python tools like Hugging Face Transformers and Google Colab's GPU computing resources, you'll create pipelines, preprocess data, and showcase sample predictions and quantitative results.

skills learned: set up training, test, and validation datasets with biomedical (MRI) images • train and evaluate Segformer and Maskformer models for brain tumor segmentation

Implement Semantic Search with ML and BERT

In this liveProject, you’ll apply premade transfer learning models to improve the context understanding of your search. You’ll implement BERT (Bidirectional Encoder Representations from Transformers) to create a semantic search engine. BERT excels in many traditional NLP tasks, like search, summarization and question answering. It will allow your search engine to find documents with terms that are contextually related to what your user is searching for, rather than just exact word occurrence.

skills learned: create an inverted index with Faiss library • encode and search documents with a sentence-transformers library • complete question-answering tasks with pretrained BERT models

Import

In this liveProject, you’ll refactor a legacy module with unpredictable functionality to make it fit for your needs. The module behaves strangely whenever you import it, and so your task is to find the problem by exploring the ways that import in Python works. Your new knowledge will allow you to fix the issues in the test module, and create safer modules and multi-module packages.

skills learned: how the import process works • what importing only one feature of a module really does • how to make sure that code only executes when you want it to

Improve Search Support

As a software engineer at Sneakers to the Max, it’s your job to create an API that the e-commerce company’s developers can use to improve search results. You’ll write Python code to create Elasticsearch queries that work best for zero, one, or multiple search terms. To enable the application to interact with the search service, you’ll implement a FastAPI endpoint. You’ll add query filters, including colors and price ranges, to narrow down the results, and you’ll add pagination for when the number of sneakers in the narrowed search results is too large to be displayed on one page. When you’re done, you’ll have created an API that empowers your developers to provide the best matches to customers’ searches and leads to increased sneaker sales.

skills learned: use different queries to select sneakers

Inheritance

In this liveProject, you’ll explore how inheritance works in Python through developing classes that might be used in a simple video game. Your game follows animal characters moving through an obstacle course, with different movement styles for wolves, bears, eagles, ducks, frogs, and trout. In order to avoid having to repeat your code between characters, you’ll create some base classes and inherit from them. You’ll see how to inherit from more than one class at a time, and discover use cases for multiple inheritance.

skills learned: using Python’s system of inheritance • developing classes for a video game • using Python's multiple inheritance to create mix-ins

Initialize Qubits and Run Quantum Circuits

Enter the world of quantum computing and discover how quantum computers solve problems in a fraction of the time. You’re a researcher for Action at a Distance Corporation, a consortium of venture capitalists who believe the time to learn about and invest in quantum computing startups is now. Your task is to learn fundamental quantum computing skills so you can evaluate proposals from quantum startups and brief the consortium’s partners. Get familiar with the Qiskit environment as you create your own Qiskit programming Python imports template, then discover features of the Qiskit ecosystem as you add code to your template. You’ll set up a quantum circuit, getting first-hand experience with amplitudes and probabilities associated with quantum states. With these fundamentals under your belt, you’ll prepare brief notes for your report to the consortium, researching which companies will be the major players in the quantum race, what types of quantum hardware they’re pursuing, and what exactly is the “quantum advantage.”

skills learned: create quantum registers and circuits and begin to program them with Python and Qiskit

Input Transactional Data

Nextstellar Corp is a media company with huge amounts of data to analyze. Some of that data is sitting in a PostgreSQL database, which is used for both authentication management and decision-making, as well as maintaining user preferences and feedback. Your boss doesn’t want that data sitting in the database—he wants it in the cloud! Moving it is exactly what you’ll be doing in this liveProject. You’ll use AWS Database Migration Service to enrich Nextstellar’s data lake with the PostgreSQL database so it can take full advantage of both modern data architecture and the AWS ecosystem.

skills learned: Using AWS Database Migration Service to move transactional data from on-premise to the cloud • Automating infrastructure using CloudFormation • Enabling on-premise to cloud connectivity • Setting up Database Migration Service replication instances and tasks • Configuring Change Data Capture (CDC) on Amazon RDS for Postgres • Using the Schema Conversion Tool • Validating a migration

Integrate Data with AWS Glue

Media company Nextstellar Corp has completed the migration of their data to the cloud—now they need to analyze it! That’s where you come in. You’ll take on the challenge of processing and integrating file and transactional data into a new cloud database solution that uses modern data architecture. You’ll use the AWS Glue tool to automate the whole process, from creating your crawlers and database to building your repository, Glue jobs, triggers, and establishing monitoring, troubleshooting, and scaling.

skills learned: Transforming raw data into standardized data using AWS Glue • Automating infrastructure using CloudFormation • Populating the AWS Glue catalog and creating Hive partitions • Creating ETL jobs using AWS Glue, including AWS Glue DynamicFrames and Spark DataFrames • Utilizing AWS Glue job observability and monitoring with AWS CloudWatch • Understanding AWS Glue job scaling concepts and configurations

Intent Classification

In this liveProject you’ll develop a chatbot that can classify a user’s intent, using the Hugging Face NLP library. Your challenges will include building the task with the Bart transformer, and building a specialized task for detecting toxic language. You’ll then develop a Telegram bot and integrate it with your toxicity classification task.

skills learned: use a transformer to create a classification tool • build a chatbot using Hugging Face library to classify user messages • use transformers to create a classifier to identify toxic messages and integrate this classification into a chatbot

Introduction to Braket API

In this liveProject, you’ll go hands-on with the basics of quantum theory to prepare yourself for building your first quantum applications. You’ll use familiar Python tools to work through the basic building blocks for constructing quantum circuits, as well as exploring quantum gates and how they transform qubits, the basic units of quantum information.

skills learned: create quantum bits (qubits) and quantum gates with NumPy • create quantum circuits with Amazon Braket SDK

Isolation Forests

Red alert! One of the energy production companies managed by Sigma Corp has suffered an outage. An investigation has led to the conclusion that the facility’s anomaly detection mechanism failed to detect early signals due to a sudden change in the distribution of the analyzed data. As a lead data scientist at Sigma, you’ll build an Isolation Forest algorithm, which is less likely than the Empirical Cumulative distribution-based Outlier Detection (ECOD) method to fail in such scenarios. To gauge how robust your method is, you’ll benchmark your algorithms against adversarial scenarios, synthetic anomalies, and standard datasets. When you’re done, you’ll have practical experience creating, using, and testing the Isolation Forest algorithm as an effective alternative to ECOD in circumstances where the data distribution changes.

skills learned: apply the Isolation Forest method • evaluate algorithms under adversarial distribution change

Iteration

In this liveProject, you’ll experiment with how iteration works in Python. You’re building a data analysis program that needs to iterate over a given number of Fibonacci numbers. To help out, you’ll construct an object that can be iterated over in a for loop or list comprehension that provides the Fibonacci series.

skills learned: build a data analysis program • use Python iteration protocol in creating iterables and an iterator using a generator function

K-means and DBSCAN Clustering

In this liveProject, you’ll use the Julia language and clustering algorithms to analyze sales data and determine groups of products with similar demand patterns. Clustering is a well-established unsupervised learning technique that’s commonly used to discover patterns and relations in data. You’ll apply k-means and DBSCAN clustering techniques to housing sales data for a retail startup, leveraging your basic Julia skills into mastery of this machine learning task.

skills learned: basic data preparation for clustering methods • clustering data with k-means and DBSCAN algorithms • evaluating and visualizing results

Latent Dirichlet Allocation

In this liveProject, you’ll use the latent dirichlet allocation (LDA) algorithm from the Gensim library to model topics from a magazine’s article back catalog. Thanks to your work on topic modeling, the new Policy and Ethics editor will be better equipped to strategically commission new articles for under-represented topics. You’ll build your text preprocessing pipeline, use topic coherence to find the number of topics, and visualize and curate the algorithm’s output for your stakeholders to easily read.

skills learned: implementing a simplified version of the LDA algorithm • preprocessing and converting a text corpus into a document-to-word matrix • generating topics using Gensim’s LDA algorithm • evaluating generated topics using Coherence and Diversity metrics • visualizing derived topics with a variety of techniques

Latent Semantic Analysis for NLP

At Finative, an ESG analytics company, you’re a data scientist who helps measure the sustainability of publicly traded companies by analyzing environmental, social, and governance (ESG) factors so Finative can report back to its clients. Recently, the CEO has decided that Finative should increase its own sustainability. You’ve been assigned the task of saving digital storage space by storing only relevant data. You’ll test different methods—including keyword retrieval with TD-IDF, computing cosine similarity, and latent semantic analysis—to find relevant keywords in documents and determine whether the documents should be discarded or saved for use in training your ML models.

skills learned: clean data with regular expressions • mathematical concepts and how and when to apply latent semantic analysis and cosine similarity

Learning Similarities

In this liveProject, you’ll determine what similarities there are between certain movies, and then build a recommendation system based on the similarities you’ve identified. Although you’ll be working with movie data for a streaming website, identifying similarities can help enhance recommendation engines for any item or product. You’ll develop a general technique for spotting similarities, then apply your learned similarities to build a recommendation engine. You’ll then enrich your model with collaborative and content-based data, and evaluate and compare your model.

skills learned: read, process, and exploit user-item data • define recommender systems based on similarities • analyze algorithm output

Leverage Embeddings

Create a powerful recommendation engine built from an ensemble of graph-based models that will help you tap into New York City’s real estate market by identifying groups of similar properties. You’ll start by working with transductive graph models (TransE and TransR) that are created specifically for knowledge graphs. Transductive learning takes observations from a specific set of training data and applies it to a specific set of test data. Next you’ll build an inductive model (GraphSAGE), which allows for generalized learning on new data (i.e. predictive modeling on previously unseen properties). Lastly, you’ll build the recommender system by using the k-Nearest Neighbor (kNN) algorithm to identify similar properties. When you’re done, you’ll have hands-on experience applying machine learning techniques to real-world knowledge graphs… and possibly a lucrative side hustle.

skills learned: build transductive graph models (TransE, TransR) with pyKEEN • build an inductive graph model (GraphSAGE) with Stellargraph

Life Expectancy Analysis

In this liveProject, you’ll perform a regression analysis on precleaned data to determine factors relating to American life expectancy. You will learn how to do a comprehensive regression analysis and how to select a model in the presence of multicollinearity. You will use the libraries developed in the second and third projects to check the validity of your final model.

skills learned: detecting multicollinearity with the Variance Inflation Factor • doing a comprehensive regression analysis with Statsmodels • visualizing relationships with seaborn • interpreting "QQ-plots," "Residuals vs. Fitted" plots, and "Scale-Location" plots

Lists

In this liveProject, you’ll utilize Python to create new data reports from a centralized system. You’ll work with raw data about the sales of various products, and massage it into a different format from which subreports can be extracted. To do this, you’ll make use of lists—one of the most common data structures in Python that are at the heart of data processing.

skills learned: list comprehensions • list slicing • ways to sort a list

ML Pipeline with Amazon Personalize

Help your company’s messenger application provide better product recommendations for its customers. As a data engineer at the company, your task is to create a machine learning (ML) pipeline using the Amazon Personalize service. You’ll use CloudFormation templates to create a repository for the required AWS infrastructure resources, and AWS Glue to transform the raw user engagement data. Using Amazon Personalize, you’ll import a dataset and create and train the Amazon Personalize ML model for your users’ recommendations. To complete the project, you’ll create a workflow to train your Amazon Personalize recommendation solution using AWS Step Functions and user engagement events. When you’re done, you’ll have designed an ML pipeline using the Amazon Personalize API that provides product recommendations that suit your users best.

skills learned: create a data lake S3 bucket using an AWS CloudFormation template (Infrastructure as Code) and prepare data using AWS Glue • design a data pipeline to create and train an Amazon Personalize solution • deploy the pipeline using AWS Step Functions and AWS CloudFormation (Infrastructure as Code)

Malware Analysis

Build a better defense! As a developer at ShareSafe, a company that provides online file-sharing-as-a-service, your task is to uplevel the security of its REST API file-transfer service by adding functionality for checking uploaded files for malware. You’ll set up a robust infrastructure for malware analysis of files stored in AWS S3 buckets, integrate the infrastructure with the REST API service, and implement VirusTotal, a threat intelligence service that provides dynamic and behavioral analysis to shared files. When you’re finished, you’ll have built a solid line of defense against known malware and provided users with a secure file-sharing experience.

skills learned: set up robust malware analysis infrastructure • integrate with third-party services

Manage Data Products

As a consultant for Messflix Inc., a movie and TV-show streaming platform, you’ll investigate and discover why Meshflix’s recommender system breaks. You’ll brainstorm options for changes that don’t break the system, explore their pros and cons, and choose and implement one of your options. Then you’ll create an internal versioning strategy to support all the great product changes Messflix has planned for the future.

skills learned: understand product thinking • manage changes in data products • prevent breaks in downstream data products

Manipulate Data Distribution

In this liveProject, you will create an imbalanced dataset from the IMDb movie dataset. Your goal is to make a dataset where positive reviews are the minority. You’ll then test a theory that if you oversample positive reviews, you could rebalance the training data to build and train a text classification model. You finish up by examining the model performance with a confusion table, and basic metrics such as precision, accuracy and recall.

skills learned: building and training a deep learning model for text classification • using sklearn module to report model classification metrics • condition based sampling technique for NumPy array

Manipulate Qubits with Quantum Gates

Action at a Distance Corporation, a consortium of venture capitalists, believes that now is the time to invest in quantum computing startups. As its resident researcher, your job is to get familiar with quantum techniques and applications. You’ll code quantum circuits involving four different quantum gates, the fundamental processing units of quantum. You’ll visualize the effects of the gates, run and measure the circuits, and use IBM Quantum Composer to demonstrate the use of the gates. With a deeper understanding of quantum, you’ll investigate major quantum use cases to help the consortium determine whether or not the startups’ proposals have commercial applications.

skills learned: investigate and program quantum circuits and quantum gates • use IBM Quantum Composer to demonstrate the use of gates • program with Python and Qiskit

Measuring Bias in a Dataset

In this liveProject, you’ll compute and chart metrics that show the key characteristics in Home Mortgage Disclosure Act (HMDA) dataset, and investigate relationships between key demographics and other features that may lead to a biased machine learning model. You’ll compute and chart “equality of outcome” bias metrics, and produce a report into your insights.

skills learned: setting up a Google Colab environment • preprocessing a dataset using one-hot encoding • charting and visualizing metrics

Measuring Bias in a Model

In this liveProject, you’ll investigate and report on whether your company’s mortgage application classifier is making fair decisions between male and female applicants. You’ll train a logistic regression classifier on the HMDA dataset, compute performance metrics with a logistic regression classifier, and then chart “equality of opportunity” bias metrics.

skills learned: setting up a Google Colab environment • preprocessing a dataset using one-hot encoding • computing bias performance metrics for logistic regression

Methods for Multidimensional Datasets

Preventing operation failures and interruptions is mission-critical at Sigma Corp. The large conglomerate of energy production companies has recently implemented a z-score anomaly detection algorithm that focuses on a single feature. Now that the algorithm has proved its value, members of Sigma have requested additional algorithms that are just as simple to use, but that can handle multidimensional data. As a lead data scientist at Sigma, you’ll implement the Mahalanobis distance (MD) method and the principal component analysis (PCA) method as you build anomaly detection algorithms for multidimensional data. To gauge the performance of your algorithms, you’ll test them against a benchmark dataset as well as synthetic anomalies generated by your own algorithms. When you’re done, you’ll have firsthand experience building anomaly detection algorithms for multidimensional datasets as well as testing anomaly detection algorithms against both benchmark datasets and synthetic anomalies.

skills learned: PCA method • Mahalanobis method • evaluate algorithms using correlated synthetic anomalies

Migrate Files to the Cloud

The Nextstellar Corp media service has a lot of data—too much data to handle on prem! In order to properly analyze all the data generated by their streaming customers, they’re migrating to the cloud. In this liveProject, you’ll be helping them. You’ll tackle the common challenge of transferring on-prem data to AWS using the handy AWS DataSync tool. You’ll use Infrastructure-as-Code to create Landing Zone Amazon S3 buckets, automate data migration, and finally prepare a summary of likely infrastructure costs for your boss to review.

skills learned: Using AWS DataSync to move files from on-premises to the cloud • Automating infrastructure using CloudFormation • Organizing and managing data in a data lake • Creating data lifecycle policies • Setting up sources and destinations for data files • Starting AWS DataSync migration tasks

Mitigate Machine Learning Bias: A Health Data Project

Unexpected bias in machine learning models reduces accuracy, produces negative real-world consequences, and in the worst cases, entrenches existing inequalities for decades. Audits that can detect and mitigate this unwanted bias are an essential part of building and maintaining reliable machine learning systems.

In this liveProject, you’ll take on the role of a data scientist running a bias audit for the World Health Organization’s healthcare models. You’ll get to grips with state-of-the-art tools and best practices for bias detection and mitigation, including interpretability methods, the AIF-360 package, and options for imputing membership of a protected class. These tools and practices are placed in the context of a broad universe of bias where geopolitical awareness and deep understanding of the history and use of your particular data is key.

skills learned: biasing and debiasing ML models • using SHAP to interpret the underlying logic behind a ML model • using AIF360 to detect and mitigate unwanted bias • auditing techniques for unknown at-risk populations

Mitigating Bias with Postprocessing

In this liveProject, you’ll use the open source AI Fairness 360 toolkit from IBM and the “equalized odds post-processing” method to post-process your model for the purpose of bias mitigation. “Equalized odds post-processing” seeks to mitigate bias by making modifications to the prediction label after a model has been trained. After charting bias metrics on a basic classifier, you’ll tune the classification threshold to explore the impact on revealed biases.

skills learned: setting up a Google Colab environment • AIF360 open source library • post-processing for fairness with the “equalized odds'' method

Mitigating Bias with Preprocessing

In this liveProject, you’ll use the open-source AI Fairness 360 toolkit from IBM to measure and mitigate bias with model preprocessing. You will chart bias metrics on a basic classifier, before preprocessing your training data with the “reweighing” method. Reweighing seeks to mitigate bias by making modifications on the training data by computing and applying a set of weights. The weights are calculated such that the training data, with weights applied, is free of discrimination with respect to a protected attribute. Once your model is preprocessed, you’ll construct a classifier that makes use of the data.

skills learned: setting up a Google Colab environment • AIF360 open-source library • preprocessing for fairness with the reweighing method

Monitoring Changes in Surface Water Using Satellite Image Data

In this liveProject, you’ll fill the shoes of a data scientist at UNESCO (United Nations Educational, Scientific and Cultural Organization). Your job involves assessing long-term changes to freshwater deposits, one of humanity’s most important resources. Recently, two European Space Agency satellites have given you a massive amount of new data in the form of satellite imagery. Your task is to build a deep learning algorithm that can process this data and automatically detect water pixels in the imagery of a region. To accomplish this, you will design, implement, and evaluate a convolutional neural network model for image pixel classification, or image segmentation. Your challenges will include compiling your data, training your model, evaluating its performance, and providing a summary of your findings to your superiors. Throughout, you’ll use the Google Collaboratory (“Colab”) coding environment to access free GPU computer resources and speed up your training times.

skills learned: work with times series data • perform image segmentation • merge satellite imagery and perform operations on raster datasets • data augmentation for boosting model training • optimize and understand model performance

Move to AWS RDS Aurora

As a lead DevOps engineer, you’ve learned that using Fargate to host a database is not ideal. Your task is to solve that problem by attaching permanent storage to the container and adding important maintenance features. To deliver this solution, you’ll set up an AWS Aurora PostgreSQL with CDK, integrate it with a Fargate service, and keep the connection between the service and the database secure with AWS Secret Manager. When you’re done, you’ll have a working end-to-end solution ready to be deployed with CDK.

skills learned: use AWS RDS Aurora with AWS Fargate • control permissions on the container • store secrets and pass them to the container without human intervention

Named-Entity Recognition

In this liveProject you’ll develop a chatbot that can extract entities from user messages using the Hugging Face NLP library. Entity extraction pulls relevant data from chunks of text. Your challenges will include building a Named Entity Recognition task with the BERT transformer, and fine-tuning it to a medical context. You’ll then integrate your tasks into a chatbot.

skills learned: use a transformer to create a named-entity recognition tool • build a chatbot using Hugging Face library to extract entities from user messages like localizations and people

Navigation

Let your robot drive itself! Your company, RoboPrototypes, is preparing to demo its autonomous beverage delivery robot to investors soon, but the technician who was supposed to drive it has quit. Since there’s no one to take his place, your CEO has tasked you with upgrading the robot with automatic navigation. You’ll set up Nav2 to load your desired parameters, implement navigation planners and controllers, and optimize navigation by tuning costmap parameters. Once you send your waypoint goals to Nav2 telling your robot where to go, it’ll be ready to quench thirsts at a food court near you.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: planners • controllers • global and local costmaps • waypoint following • tuning Nav2 parameters

Nested Lists

In this liveProject, you’ll play the role of a programmer who has been hired by a real estate developer to help determine what a fair market price for rent should be across a number of owned properties. The good news is that the real estate developer has gathered data into a spreadsheet (in a CSV file). The bad news is that the spreadsheet is so large that they can’t even open it in a typical spreadsheet program, let alone do any sort of analysis with it. They’ve asked you to develop an interactive program that will perform some basic exploration of the data. You’ll use highly popular Python to tackle this challenge, and gain valuable experience processing CSV files and sorting lists as well as working with files, strings, and command-line arguments.

skills learned: develop a basic text-based spreadsheet program • read command-line arguments and perform file I/O • perform common string operations in Python • append items to Python lists • sort lists in Python using built-in routines

Neural Networks

In this liveProject, you’ll build neural nets for a bank that are capable of classifying the numbers written on deposit slips. Customers often write out their important account details by hand, and you need to create a model capable of effectively reading their handwriting. You’ll use a feedforward network trained with backpropagation to identify hand-drawn digits with a high percentage of accuracy. Plus, this project contains an optional bonus milestone! Further your knowledge of neural networks by using the TensorFlow and Keras libraries to create an alternative and more flexible version of your first model. You’ll get hands-on experience with these tools' standard libraries, and with your new model you will be able to easily analyze the network's success rate and experiment with different activation functions.

skills learned: neural network concepts • feed-forward neural networks • backpropagation

Neural Topic Models

In this liveProject, you’ll use the neural network-inspired Contextual Topic Model to identify and visualize all of the articles in a scientific magazine’s back catalog. This cutting-edge technique is made easy by the OCTIS (Optimizing and Comparing Topic Models is Simple!) library. Once you’ve established your text-processing pipeline, you’ll use coherence and diversity metrics to evaluate the output of your topic models, tune your neural network’s hyperparameters to improve results, and visualize your results for printing on posters and other media.

skills learned: exploring the vector representations produced by BERT • preprocessing text documents • using the Contextual Topic Model • performing hyperparameter tuning • evaluating topics using Coherence and Diversity and visualizing generated topics

Non-negative Matrix Factorization

In this liveProject you’ll use scikit-learn’s non-negative matrix factorization algorithm to perform topic modeling on a dataset of Twitter posts. You’ll step into the role of a data scientist tasked with summarizing Twitter discussions for the customer support team of an airline company and use this powerful algorithm to rapidly make sense of a large and complex text corpus. You’ll build a text preprocessing pipeline from scratch, visualize topic models, and finally compile a report of support topics for the customer services team.

skills learned: Implementing a simplified version of the NMF algorithm • preprocessing and converting text corpus into a document-to-word matrix • generating topics using scikit-learn’s NMF algorithm • evaluating generated topics using Coherence and Diversity metrics • visualizing derived topics with a variety of techniques

Nonogram

In this liveProject, you’ll use Python to build a nonogram puzzle generator that creates a printable puzzle pack from the command line. Nonograms are puzzles that reveal an image on a grid, based on numbers along the grid’s edge. Most programmatic implementations solve nonograms, but yours will generate them. To do this, you’ll code your script, relying on two essential data structures: the pandas DataFrame and the NumPy array. You’ll also experiment with image libraries including Pillow and pixelate, and gain a better understanding of basic image processing concepts.

skills learned: image manipulation • NumPy • table styling • file I/O • HTML • CSS • debugging • Pillow • pixelate • regular • expressions

Object Detection

In this liveProject, you'll spearhead the development of AI-aided surveillance software for construction site supervision. You’ll build two computer vision applications capable of detecting construction vehicles and their types across a large worksite and a more powerful model that can detect building defects such as cracks and fissures. Start by working with a pre-trained DETR model, then explore the Roboflow platform to assist you as you create a multi-class object detection dataset from multiple datasets with non-identical classes. With these datasets, you will train different transformer models for object detection to identify construction vehicles and cracks in buildings.

skills learned: import computer vision datasets from other platforms like Roboflow• merge two or more object detection datasets and their annotations on Roboflow • set up training, test, and validation datasets for object detection • train and evaluate three variants of DETR models for construction vehicle detection

Objects and Methods

In this liveProject, you’ll take on the task of extending a pricing application. You’ve just hit your first speed bump of needing to add functionality to the class that manages prices. However, that class is used elsewhere and you can’t get permission to change it. Your challenge is to implement a “monkey patch”—adding methods to the class at runtime—by experimenting with Python-bound methods.

skills learned: add functionality to a pricing application • create a class using type() • add bound methods to a class • monkey patch a class after creation

Observability

Put on your platform architect hat! You’re a member of the development team at Piper Data Concepts (PDC), and your client is looking to modernize its workflow. An existing benchmarked development environment, made up of Kafka, Python, and Postgres, is at your disposal. Now it’s time to start conceptualizing the new and improved workflow. You’ll use Kafka to create an event-driven data pipeline, review and understand business requirements, use Python Poetry to package the project, write Python code using the Faust library to communicate with Kafka, and store the consumed data in a PostgreSQL database.

skills learned: Faust • architecture • Kafka topics • writing to a database

Optimize Mapping

You’re a software engineer for Sneakers to the Max, an e-commerce company that sells sneakers. Its CEO wants to improve the speed and quality of the company’s search engine in hopes of boosting sales. Your task is to optimize the mappings of the Elasticsearch index. You’ll use the Kibana developer console to analyze the current mapping and create a new custom analyzer. To improve search matches—and customer experience—you’ll use the Elasticsearch field mappings to support multiple uses of the same field. You’ll modify the mappings to help customers find the sneakers they’re looking for, even if their search terms are incomplete or contain typos. To further improve CX, you’ll add synonyms to the Elasticsearch configuration to handle searches that use different terms for the same concept.

skills learned: mapping of different datatype fields in Elasticsearch • evaluating mapped datatype fields using analyzer tools

Orchestrate an ETL Pipeline

Nextstellar Corp needs you to tackle a big challenge: completing their cloud migration by rebuilding their historical data lake as a data layer in the cloud. You’ll implement an effective and automated data orchestration framework for the ingestion and transform and curate layers, using best practices for Infrastructure-as-Code to automate your data layer. Finally, you’ll establish a monitoring system that will automatically alert you to any issues or problems that might crop up.

skills learned: Using AWS Step Functions and AWS Glue workflows to orchestrate data pipelines • Automating infrastructure using CloudFormation • Monitoring data in a data pipeline.

Organizations Service Control Policy

In this liveProject, you’ll leverage organizations service control policies to protect administrative and security resources in the accounts used by development teams where these teams have privileged access. Specifically, you will add automation to restrict access of local administrators (and intruders!) to any AWS CloudTrail and Amazon GuardDuty resources located in the accounts. Without these protections, the security controls we have put in place for audit, forensic analysis, and intrusion detection can be interfered with by the local administrators.

skills learned: develop Security Control Policies • develop and test boto code • invoke the IAM Policy Simulator

Pandas for Feature Extraction

Master the basic methods for handling most real-world scenarios as you play the role of a data scientist in an online recruiting tech company or a large organization’s HR department. Using pandas, you’ll process, extract, and transform numerical, categorical, time series, and text data into structured features that are ready for data analysis and ML model training. When you’re done, you’ll have hands-on experience working with most data types you’ll find in the real world, as well as useful skills for extracting and engineering features.

skills learned: feature engineering with pandas • cleaning, standardizing, and transforming data with pandas

Personalized Diagnosis of Symptoms

In this liveProject, you’ll apply the federated learning machine learning paradigm to compute a personalized infection risk diagnosis for COVID-19. Federated learning is an ML paradigm that learns from decentralized data via distributed computing environments. The risk diagnosis will come from a tailored classifier that is trained separately for each individual, by combining gradient descent with a network averaging method. You’ll then develop a federate learning algorithm that can be implemented by scalable passing messages over the contact network.

skills learned: use NetworkX to jointly represent network structured data and models • learn the weights of a linear classifier using gradient descent • implement a Federated Learning algorithm by combining gradient descent with a simple averaging scheme

Personalizing Discounts

In this liveProject, you’ll use causal inference to investigate data on randomized discounts and determine if an e-commerce company should offer personalized discounting. You’ll estimate a different treatment effect for each customer in the hopes to see if some are positive, and figure out which customers should get what discounts. You’ll utilize Python and machine learning to build this personalization system, and implement a causal model for personalization.

skills learned: implement causal models for personalization • evaluate the performance of your causal model using experimental data

Plotting with Seaborn

You’re a data analyst at Global Consensus Bureau, and your manager has asked you to determine how factors like education, hours worked per week, gender, and age relate to citizens’ incomes. Using pandas, you’ll pre-process data from a survey conducted by the U.S. Census Bureau. Then, you’ll create different charts to identify statistical relationships in data, using Python’s seaborn library, which provides custom functions for visual elements. When you’re done, you’ll know how to use seaborn to create visually appealing charts that reveal accurate and useful insights.

skills learned: visualize relations in categorical data • visualize relations in numerical data • visualize pairwise relationships

Predicting Disease Outbreaks with Time Series Analysis

Time series analysis is an essential tool for data forecasting, allowing data analysts to make predictions about the future events and track relationships between data. In this liveProject, you’ll utilize the Python data ecosystem and time series analysis to analyze the spread of the COVID-19 virus in different parts of the globe.

Your goal is to make near-future predictions about virus spread based on your available data. You’ll start with an exploratory data analysis into the types of data you have access to, establishing the kind of questions it can reasonably answer. You’ll then develop an ARIMA model for time-series forecasting. Finally, you’ll develop an interactive Voilà dashboard using ipywidgets that will allow stakeholders to access and understand your analysis.

skills learned: time series forecasting with ARIMA • univariate and bivariate analysis • data visualization

Predicting Loan Defaults Using scikit-learn and H2O

The Python data science ecosystem is a powerful and open-source toolset utilized daily by thousands of data scientists and machine learning engineers. But with so many Python machine learning libraries to choose from, which tool works best for your needs?

In this liveProject, you’ll go hands-on with the scikit-learn and H2O frameworks, using them both to build working machine learning classifiers. You’ll use raw financial data and the tried-and-true random forest model to predict the chance of financial loan defaults. Once you’ve built your models, you'll compare implementations to find out which works best and evaluate your results against existing hard-coded tools.

skills learned: exploratory data analysis • working with pandas DataFrames • feature engineering • machine learning modeling with random forests • optimizing machine learning • model evaluation and comparison • deploying a model in a Python module

Predicting Urban Events with scikit-learn

A big city receives thousands of daily calls to its emergency services, reporting everything from illegal parking to life-threatening emergencies. In this liveProject, you’ll take on the role of a data scientist called in to advise on how the city can better allocate resources for these safety events. Your challenge is to create a machine learning model that can predict when and where different emergencies will occur. To do this, you’ll analyze data to identify trends, build and enhance a predictive model, and make your model explainable with model interpretability tools. You’ll also perform checks to ensure that the model won’t lead to bias or discrimination, and tweak your model so it can account for major lockdown events such as a pandemic.

skills learned: using machine learning to analyze and visualize location-based and time dependent data to make predictions • interactive data visualization • evaluating risks of bias and discrimination in production models

Prepare Tabular Data

In this liveProject, you’ll use the Python tools pandas and Folium to prepare the Airbnb dataset for training a deep learning model. You’ll learn Python concepts that ensure the application you build on this data is robust and maintainable, and implement geospatial and visualization techniques to illustrate the geographic distribution of Airbnb rentals.

skills learned: manipulate tabular data • create geospatial visualizations

Prepare Time Series Data

In this liveProject, you’ll create baseline models with Naive and sNaive methods for time series forecasts that you can use as a point of comparison for other models. Taking on the role of a data scientist for a large retail company, you’ll go hands-on to prepare sales data, create the baseline models, and optimize a Prophet model to compare against your baseline.

skills learned: build baseline models with Naive and sNaive methods • set up and optimize a Prophet model as an alternative model for comparison • evaluate various models using weighted MAE and MAPE

Preparing a Quantum State

In this liveProject, you’ll step into the shoes of a quantum software engineer and develop a library for state preparation routines. Preparing quantum systems in arbitrary superposition states is a vital step of quantum algorithms. Luckily, the Microsoft Quantum Development Kit libraries offer multiple state preparation routines. You’ll implement a single-qubit state preparation routine and a recursive multi-qubit state preparation routine, before implementing the complete multi-qubit state preparation routine. Finally, you’ll write unit tests for your implementation.

skills learned: developing quantum algorithms to match specified tasks • simplifying code development with the Microsoft Quantum Development Kit • running quantum code simulations

Preprocess Data in Parallel

The company you work for, which offers lifestyle products, has enjoyed success for several years, but management has decided it’s not competitive enough. Your task is to identify current market trends and new niche markets for the company’s lifestyle products. Using Python and Ray, you’ll build a web scraper that will load and save multiple web pages concurrently. To preprocess the data, you’ll read each of your locally stored pages, split them into sentences, tokenize each sentence with Hugging Face tokenizers, and store your tokenized documents in a new (pickled) format in your file system. When you’re finished, you’ll have leveraged Ray to preprocess a large amount of data while bypassing Python’s notorious concurrency limitations. The data science team will thank you for helping minimize the data preparation time, and the management team will thank you for helping the company sharpen its competitive edge.

skills learned: make your Python code concurrent or parallel • build a web scraper • pre-process texts for NLP models with Hugging Face transformers

Principal Component Analysis

Step into the role of data scientist at Finative, an analytics company that uses environmental, social, and governance (ESG) factors to measure companies’ sustainability, a brand new, eco-focused trend that's changing the way businesses think about investing. To provide its clients with the valuable insights they need in order to develop their investment strategies, Finative analyzes a high volume of data using advanced natural language processing (NLP) techniques.

Recently, your CEO has decided that Finative should increase its own sustainability. Your task is to develop a method to optimize the runtime for the company’s machine learning models. You’ll apply principal component analysis (PCA) to the data in order to speed up the ML models. To classify handwritten digits and prove your theory that PCA speeds up ML algorithms, you’ll implement logistic regression with scikit-learn. You’ll use the explained variance ratio to gain an understanding of the trade-offs between speed and accuracy. When you’re done, you’ll be able to present your CEO with proof of PCA’s efficiency in optimizing runtime.

skills learned: algorithm optimization with principal component analysis (PCA) • matrix manipulation with NumPy • nuances of scikit-learn's PCA library

Prophet Model Incorporated with Bayesian Analysis

In this liveProject, you’ll build a Prophet model that can forecast airline passenger numbers using data from the DataSF portal. The hotel you work for believes that analyzing the travel trends of US customers will help them forecast potential travel to Europe, and bookings in the hotel. You’ll enhance your model with "changepoints" that mark a significant change in trends, and make adjustments so your model can account for uncertainty in the trend and seasonality components.

skills learned: implementing prophet models • configuring changepoints • analyzing time series data with uncertainty interval configuration

Publish Data Products

Messflix Inc., a movie and TV-show streaming platform, wants to build a recommendation system for its movies and shows, but currently, its data landscape is too complex. As a consultant, your task is to implement a data mesh for an improved, accurate flow of its data. Using Python and JSON, you’ll help the data engineering teams sift more easily through Messflix’s data by creating separate, structured data products that can be pushed to the central data platform. From the organized data products, you’ll create a list of recommended movies, tailored to Messflix’s customers’ preferences.

skills learned: understand distributed domain ownership • produce data inside a data mesh • produce derived domain data products

Push Data into Data Products

Messflix Inc., a movie and TV-show streaming platform, is implementing a data mesh. So far, it has a self-serve data platform prototype where development teams can register their domain data products. As a consultant, your task is to build on that basic platform prototype with additional functionality: You’ll write a script in Python that will enable the development teams to push fresh data into their existing data products, write a function that adds support for versioned data, and implement a function that automatically calculates specific metadata (like row count and latest timestamp), then prints it to the screen. When you’re finished, you’ll have built a well-functioning, feature-rich, self-serve data platform, and be familiar with the requirements data-producing and data-consuming teams face daily and how to fulfill them.

skills learned: build a self-serve data platform • build a small data catalog • consume data inside a data mesh • store data products

Python and Kafka

Step into the role of a developer at Piper Data Concepts (PDC), a company that specializes in helping Fortune 1000 companies improve their workflows. Your task is to review the 15-year-old workflow architecture of one of your clients, Trade Data Systems. You’ll identify issues and bottlenecks, then determine what’s needed to transform its workflow into a more modern, responsive architecture. To accomplish this, you’ll set up a development environment with Docker using Kafka, Python, and Postgres. As you go, you’ll deploy a Kafka cluster and write Python code using the Faust library to seamlessly process pre-defined business events.

skills learned: Python packaging • messaging • containers

Q&A Using Vector Databases

In this liveProject, you'll join InfoHub, an AI startup disrupting corporate knowledge management. They aim to unlock a company’s knowledge base through conversational Q&A-style user interfaces that use breakthrough language models. You'll leverage LangChain, a framework optimized for integrating LLMs into apps, to integrate InfoHub's data, vector stores, and language models into a single solution. You’ll prepare your data, create a vector store to embed your documents, and then use LangChain to combine it with an LLM.

skills learned: use LangChain for questions answering • create and populate a vector store with your own data

Quantum Communication

In this liveProject, you’ll implement superdense encoding and use the unique properties of qubits to build a communication system between two friends—Alice and Bob. You’ll prepare qubits into a Bell pair, and then utilize quantum gates to implement a messaging system. You’ll then prepare the quantum circuits that allow Bob to receive and read Alice’s messages.

skills learned: create a quantum circuit to communicate information between two parties using the Amazon Braket SDK • prepare two qubits and make them into a Bell pair • implement a messaging scheme with quantum gates • send messages using qubits

Quantum Random Number Generator

In this liveProject, you’ll use the Amazon Braket quantum simulator to generate a quantum random number generator. Generating truly random numbers is an essential aspect of cryptography, and the seed generation process in a public/private key infrastructure. These random numbers can be used as the basis for further cryptographic work that easily outpaces the pseudo-random capabilities of traditional computing.

skills learned: create a quantum circuit that solves a problem by creating a truly random number with Amazon Braket SDK

Question Answering

In this liveProject you’ll develop a chatbot that can answer its user’s questions, using the Hugging Face NLP library. Your challenges will include building the task with DistilBERT transformer, and experimenting with other transformer models to improve your results. Once you’ve built an accurate NLP model, you’ll develop a Telegram bot and integrate it with your Question Answer task.

skills learned: use a transformer to create a question-answer tool • build a chatbot using Hugging Face library to answer user questions using Wikipedia data

Question Answering with Deep Learning

In this liveProject, you’ll step into the role of a data scientist working for an investment firm. Your company wants to make sure their investments meet European Union guidelines for environmental sustainability. That’s where you come in.

The EU taxonomy for sustainable finance is big, complex, and confusing. Your bosses need a program that saves them from searching through hundreds of pages whenever they have a query. You’ve been tasked with building a machine learning model that can pose certain questions to the EU guidelines, and return reliable answers.

Your challenges will include extracting text data from the EU taxonomy document, and matching environment questions with the corresponding paragraph in the guidelines. You’ll then set up a pretrained transformer Question-Answering model, evaluate its performance, and combine it with your question-paragraph model for an end-to-end solution. When you’re done, you’ll have an interface into which you can type a sustainable finance question and receive the correct answer from the EU guidelines.

skills learned: deep learning for NLP • information retrieval • extracting text from PDFs • working with transformers and auto-encoders • working with word and paragraph embeddings

Real-World Inference

Behind the scenes on websites like Amazon, Netflix, and Spotify, models make predictions on thousands of items every day. Then, based on what they’ve learned, they choose only the best recommendations to display for every individual user. In the real world, performing thousands of predictions one by one, as in a notebook-only model, would be highly inefficient. In this liveProject, you’ll reconfigure the models you implemented in the previous project to accept a list of items for each user and then evaluate all items at once—choosing the best recommendations much more quickly and efficiently.

skills learned: reconfigure a “notebook-only” recommendation system to a model capable of handling many-items per user • use linear-algebra to combine networks with differing dimensionality

Recognize Speech Commands with Deep Learning

Voice assistants and speech-based applications need a reliable understanding of human commands. Deep learning systems trained on audio data are a great way for models to learn directly from sound and easily classify what humans are telling them.

In this liveProject, you’ll become a data scientist prototyping a speech command system for a medical organization. Your company’s application will allow hospital patients to adjust their room TVs with their voice, and you’ve been tasked with building the core: a machine learning pipeline that can label audio inputs with given command words. You’ll tackle preprocessing and feature engineering methods for audio signals, expand your existing machine learning capabilities with audio data, and get hands-on experience with TensorFlow, Keras, and other powerful Python deep learning tools.

skills learned: performing digital audio processing • extracting spectral features from raw audio data • building deep learning architectures for audio

Regression Diagnostics

In this liveProject, you’ll build your own Python data library to check three of the assumptions of the regression model: normality, linearity, and constant variance. Confidence in your regression models depends on how well you have satisfied these assumptions. Once you’ve developed functions and plots that can check these assumptions, you’ll master techniques for correcting them. With this library, you will expand your data science toolbox with important diagnostics tools that will allow you to be confident in your results.

skills learned: checking the assumptions of the regression model • programming "QQ-plots," "Residuals vs. Fitted" plots, and "Scale-Location" plots • visualizing relationships and distributions with seaborn

Regression Discontinuity Design

In this liveProject, you’ll use regression discontinuity design (RDD) as a form of natural experiment that works as an alternative to A/B testing. RDD measures treatment effects at points of discontinuity to get an idea of the effectiveness of a program without needing to rigorously A/B test it. You’ll use this technique to assess the discounting strategy of an e-commerce company, in a way that will allow you to start applying causal inference to other practical problems.

skills learned: spot discontinuities in data • use RDD techniques to estimate the impact of discount on profits

Regression Using GLM and DecisionTree

In this liveProject, you’ll use the Julia language to build a regression-based machine learning model that can predict median house value in a neighborhood. You’ll start out with a simple linear regression model to give you a baseline value for quality metrics created with Julia’s package for Generalized Linear Models. You’ll then tune and assess a random forest model, and compare and contrast the two approaches to pick the best results.

skills learned: build linear regression and random forest models • evaluate models and explore their output • save model to the file for further reuse

Reinforcement Learning for Self-driving Vehicles

In this liveProject, you’ll develop an AI driving agent that can simulate independent driving. Your agent should be able to handle changing lanes, navigating roundabouts, and even self-parking. In order to create this complex agent, you’ll use powerful reinforcement learning techniques including model-based learning, model-free learning, and simulation-search/planning algorithms. Go hands-on to train your AI in AI Gym on Google Colab, improve your model’s performance, and provide professional-level documentation of your work for your colleagues.

skills learned: working with OpenAI Gym • developing reward-shaped training agents • using rl-agent and stable-baselines packages to directly train models

Retrieval and Ranking

Real-world recommendation systems work with millions of users and billions of items. To handle this massive scale, recommendation systems tend to be divided into two types of models (retrieval and ranking), running one after the other, narrowing down the set of items each time. In this liveProject, you’ll implement a retrieval model using TensorFlow Recommenders, combine it with a ranking model, then create a fully functional pipeline going through both models, using a Feature Store. Finally, you’ll explore the scenario where you can’t (or choose not to) run both retrieval and ranking models online in real-time, leveraging the Feature Store once more to use the retrieval model offline.

skills learned: split a single recommendation system to two essential parts, retrieval and ranking, using TensorFlow Recommenders • understand what Feature Stores are and how to use them

Run Circuits on Quantum Hardware

Action at a Distance Corporation, a consortium of venture capitalists, believes that the time to invest in quantum computing startups is now. As the consortium’s researcher, your task is to gain knowledge of quantum computing capabilities.

You’ll explore matrix representations—mathematical underpinnings of quantum computing—and examine quantum circuits that use different arrangements of gates to achieve identical functionality. You’ll upgrade your quantum toolbox with phase kickback, an essential component of many quantum algorithms, and you’ll work with Bloch spheres, a way of visually representing qubits. You’ll then use two different methods to run a circuit on IBM quantum hardware, a task that will be commonplace in the not-so-distant future.

skills learned: quantum gates as matrices • circuit identities • phase kickback • Bloch spheres • run programs on IBM quantum hardware systems on quantum computers

Run Two-Qubit Circuits

Get tangled up in quantum computing! Action at a Distance Corporation, a consortium of venture capitalists, believes the time to invest in quantum computing startups is now. As the consortium’s resident researcher, your task is to understand the power of quantum computing.

You’ll learn to recognize normalized quantum states of two qubits, a key skill in implementing quantum algorithms. You’ll get familiar with the Controlled Not (CNOT) gate, vital to the construction of important, entangled states, and you’ll observe what Einstein called “spooky action at a distance” as you program the (famous!) entangled Bell states. Using the IBM Quantum Composer, you’ll visualize and animate quantum circuits, gaining insight into handling gates and their outcomes.

skills learned: programming circuits involving 2 qubits • quantum gates • Bell states • IBM Quantum Composer

Run a Real-Time ML Model

EKKo Inc., the machine learning consultancy you work for, has been building an embedded system to enable deaf or hard of hearing people to participate in online meetings and events on their mobile devices. As a data scientist, your task is to complete the system by enabling it to detect American Sign Language (ASL) in real time. You’ll define a CNN model, train it using the existing ASL dataset, and evaluate the model’s accuracy. To optimize training speed and quality, you’ll fine-tune the model’s hyperparameters using the Keras Tuner. You’ll enable the model to be quantization-aware with the TensorFlow Model Optimization Toolkit (MoT), optimizing size and CPU consumption while maintaining model accuracy. To complete the project, you’ll connect the quantized model with a live video stream and train it on the embedded system. When you’re done, you’ll have a fully functional quantized CNN you can run on an embedded system that successfully detects and transcribes ASL in real time.

skills learned: set up a real-time training algorithm on an embedded board • train, evaluate, and tune hyperparameters on an embedded board • workflow of model tuning from the desktop to the embedded board

Scaling and Operation

You and your DevOps team have successfully built an application and an automated deployment pipeline for it. But before you can launch you’ll need a solid operating plan that addresses how you’ll monitor and troubleshoot your new service in production. In this liveProject, you’ll add metrics and a CloudWatch Container Insight dashboard to your Fargate service. You’ll also enable autoscaling to adjust according to CPU usage, and you’ll structure logging to be easily searched. Lastly, you’ll use Python to set up alarms so you’ll know when something goes wrong.

skills learned: use AWS RDS Aurora with AWS Fargate • control permissions on the container • pass parameters and secrets to the container

Scheduling

In this liveProject, you’ll take on the role of a project manager for a construction company and utilize Python to help schedule your construction’s tasks. You’ll utilize tried and tested methods like topological sorting, PERT charts, and Gantt charts to create a task scheduling tool that can both track deadlines and automatically update delivery dates for when deadlines are missed.

skills learned: using Python to parse data in files • finding critical paths • drawing with tkinter

Search Algorithms

The real estate developer you’ve worked with previously is so pleased with your work that they’ve asked you to write a new program that allows them to zero in on specific data in sorted reports. Their ultimate goal is to determine what kinds of homes fall into different ranges of interest. Using Python, the csv library, and a custom ADT (that you’ll design), you’ll implement multiple distinct search algorithms as you create a program that allows your client to run many queries— and be fast and efficient, despite the large size of the dataset.

skills learned: design an ADT that abstracts over a dataset and implement it for sorted and unsorted Python lists as well as balanced and unbalanced binary search trees • implement a routine to search through the data • check the correctness of your code using provided unit tests

Semi-Supervised GANs for Melanoma Detection

In this liveProject, you’ll utilize PyTorch and powerful semi-supervised learning techniques to construct an advanced image classifier that can tell whether a 32x32 pixel photo of a mole is melanoma-positive—despite working with a very small labelled dataset. You’ll set up your image preprocessing pipeline, feed data into your PyTorch model, and then train a semi-supervised GAN model on both labeled and unlabeled datasets.

skills learned: generative modeling with unsupervised GANs • semi-supervised learning with GANs

Sensors and Sensor Fusion

You’re a programmer at the RoboPrototypes company, and its “dribot”—a prototype of a robot that will serve drinks to customers at a food court—has been running into things (including the CEO’s shin). Using Gazebo 11 and RVIZ2, you’ll add LiDAR (light detection and ranging) and IMU (Inertial Measurement Units) sensors that will provide a view of the environment from the robot’s perspective. To make the most out of the sensor data, you’ll fuse information from multiple sensor sources to better estimate where the robot is in the world.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: basics of sensors (encoders, IMUs, LiDARs, ultrasonic sensors) • URDF/XACRO • create a robot description • simulate sensors • sensor fusion

Sentiment Analysis and Natural Language Processing for Marketing

In this liveProject, you’ll step into the role of a Natural Language Processing Specialist working in the Growth Hacking Team of a new video game startup. Your team wants to massively accelerate your company’s early growth by acquiring huge numbers of customers at the lowest possible cost. To help tailor marketing messages, your boss has asked you to map the market and find out how customers evaluate your competitors’ products. Your challenge is to create a sentiment analyzer that will give a deeper understanding of customer feedback and opinions. To do this, you’ll need to download and create a dataset from Amazon reviews, build an algorithm that will determine whether a review is positive or negative, evaluate your analyzer's performance against star ratings, and lay out your findings in a report for your manager.

skills learned: perform sampling from imbalanced data • dictionary-based sentiment analysis • analyze reviews with deep learning • compare classifier performance

Sentiment Analysis of Video with AWS Services

In this liveProject, you’ll make use of Amazon Web Services tools to convert customer review videos to text for sentiment analysis. You’ll step into the role of a developer at a startup that offers a video alternative to feedback surveys, and help scale up your current platform so that it can handle thousands of incoming videos.

You’ll start creating your highly scalable solution with secure uploading, then build a video processing pipeline using event-based architectures. You use AWS Elastic Transcoder and Amazon Transcribe to transcribe video to text, and Amazon Comprehend to run sentiment analysis to determine whether feedback is positive or negative.

skills learned: create a serverless project and event-driven architecture with AWS Chalice • video manipulation with AWS Transcoder and Amazon Transcribe • sentiment analysis with Amazon Comprehend

Serve a Compound Model

The company you work for, which provides a news feed aggregator, is plagued with an influx of hoaxes that are putting its reputation in jeopardy. The data science team has already trained a set of complex natural language processing (NLP) models to distinguish real news from fake news. Your task is to build a service, using Ray, that exposes the endpoint that returns the JSON object categorized as either a hoax or news. Then, you’ll optimize the service for performance and speed, enabling it to perform more parallel operations and use as many GPUs as possible. When you’re finished, you’ll have firsthand experience using some of Ray Serve’s advanced features for serving and optimizing a compound model—and you’ll have kept your company’s reputation safe.

skills learned: use Ray Serve • use FastAPI • NLP text preprocessing • use Hugging Face transformers to transform text

Serve a Model

Step into the role of a machine learning engineer working for a healthcare company that provides software to hospitals. One of your clients, a national health provider, has asked your company to provide software that predicts heart failure in patients. Using scikit-learn, you’ll develop a model that uses linear regression on a public Kaggle dataset containing heart failure data. Using Ray Serve, you’ll first deploy a function that accepts a wide range of parameters, then serve your model and provide functionality for multiple concurrent requests. When you’re done, you’ll have learned to use the Ray framework to serve your model through a webpage and helped your client save lives by using its patients’ parameters to predict imminent heart failure.

skills learned: use Ray Serve • use FastAPI integration with Ray • deploy a scikit model with Ray Serve

Serverless Transformation

Nextstellar Corp is very excited—they’re taking all their data to the cloud! They’ve recently migrated and have an early-defined data lake solution. The CEO has approached you to deliver the next step of their cloud data process: using AWS Glue to apply the transformation logic and store the curated data in Amazon S3. You’ll utilize Jupyter Notebooks to curate and standardize your data sets—crafting, organizing, and managing datasets to ensure they are easily accessible and usable—then design a CI/CD pipeline to test and deploy code with a single push after completion.

skills learned: Transforming standardized data into curated data using PySpark • Automating infrastructure using CloudFormation • Developing ETL jobs using AWS Glue Jupyter Notebook • Implementing CI/CD for code release in AWS Glue Jobs • Refactoring Jupyter code and deploying through the CI/CD pipeline

Service Control Policy

QryptoTremolo, a startup that’s been developing a set of next-generation financial services, has already taken steps to provide visibility into its AWS accounts and set up AWS GuardDuty to detect intruders. But managers of the development teams have administrator access to these accounts, and despite being a dangerous anti-pattern, that’s not going to change anytime soon. Your task is to automate the configuration of organizational service control policies that restrict access of local administrators (and intruders) to any AWS Config and EventBridge resources located in the AWS accounts. You’ll write code to query the IAM (Identity and Access Management) policy simulator in order to gauge API calls’ ability to tamper with resources. You’ll also develop end-to-end tests that prove whether an SCP change will behave as expected, and build a pipeline to codify the change control process for the SCP.

skills learned: organizations service control policies • AWS CodeBuild • AWS CodeCommit • pytest • CloudWatchEvents • Boto

Set Up a CD Pipeline

A working application is good; an automatically deployable application is great! In this liveProject, you’ll create a continuous delivery (CD) pipeline for your application using CDK. Then, you’ll deploy your application in a single account as well as in multiple accounts using CDK’s pipeline module. Along the way, you’ll learn to spot and evade possible pitfalls.

skills learned: create a pipeline to automate application building and deployment to AWS Fargate • use AWS CDK to automate deployment of a CD pipeline

Shortest Paths

In this liveProject, you’ll use Python algorithms to help find the shortest path between stops for a fleet management company. With trucks driving over 10 million miles per year, even slight time saving can be a huge cost reduction. You’ll build network classes, save, restore, and draw networks, use network algorithms to find shortest paths between locations, and lay the groundwork for future network applications.

skills learned: defining classes that represent network nodes and links • saving and restoring networks • automatically building test networks • finding shortest paths in a network

Simple Classification

This liveProject puts you on the front lines at an indie bookstore, where poorly scrawled notes threaten to bring sales to a standstill. Learning classification algorithms from the ground up, you'll use four different and surprisingly simple classifiers to develop an AI solution that turns illegible scribbles into recognizable numbers.

skills learned: simple classification algorithms • K-nearest neighbors (KNN) • K-means • Naïve Bayes • hierarchical clustering

Simulate a Robot

The future of the RoboPrototypes company is in your hands. It’s your first day at the company, which specializes in bespoke robotic solutions. The CEO has asked you to develop a prototype of a mobile robot that will serve drinks to mall food court customers. An important client will decide whether they’ll provide follow-up funding after they see your prototype in simulation. Working with Gazebo 11 and employing ROS best practices, you’ll create a model for a differential drive robot that meets the provided specifications. Next, you’ll simulate the robot, implement a basic teleoperation node for controlling it, and add velocity multiplexing for safety purposes.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: basics of URDF and XACRO modeling • load a robot model in RVIZ2 and Gazebo 11 • send Twist messages to the robot (and some of the safety implications involved)

Simultaneous Localization and Mapping

Keep tabs on your robot. RoboPrototype, the company you work for, is preparing to demo its autonomous beverage delivery robot for investors in a food court next month. Your CEO has tasked you with setting up SLAM (Simultaneous Localization and Mapping) on the robot, making it possible to locate the robot at all times. Using the Slam Toolbox, you’ll equip the robot with the ability to create a map of its environment as well as pinpoint its location on the map. You’ll also get firsthand experience mapping on real data captured on a Turtlebot3 Burger, a widely used mobile robot platform.

The version of Gazebo used in this project reached end of life in January 2025.

skills learned: map with slam_toolbox • localization with slam_toolbox • SLAM in the bag

Site Reliability Engineering

As an enterprise architect at Padre Inc., which runs its cloud operations on AWS, it’s up to you to manage multi-cloud operations for Tiddler Inc., a startup using Google Cloud that Padre has acquired for its SaaS project. Your task is to implement error budgeting to achieve a predefined service-level objective (SLO) and balance the deployment of new features with the reliability of the services in production. Using tools including Elasticsearch, Kibana, and Logstash, you’ll create an error budget policy and you’ll calculate burn rate, which indicates how quickly the budget is expended. You’ll use Kibana to implement a dashboard that uses burn rate data when creating the alerts product owners rely on for ensuring that service reliability meets service-level agreements (SLAs) when releasing new features. When you’re finished, you’ll have hands-on experience with valuable site reliability engineering (SRE) skills and concepts that you can apply to real-world projects.

skills learned: implement Server Reliability Engineering (SRE) with SLIs, SLOs, and SLAs