Create a Data Platform for Real-time Anomaly Detection

prerequisites: intermediate Python • basics of pandas • basics of Matplotlib • basics of scikit-learn • basics of Docker • basics of exploratory data analysis • basics of machine learning • basics of software engineering
skills learned: design a data platform architecture consisting of several Dockerized components • train an anomaly detection model with scikit-learn and deploy it in a web service • monitor the web service and anomaly detection model with Prometheus and Grafana

Juan De Dios Santos Rivera

7 weeks · 5-7 hours per week · INTERMEDIATE

filed under

placing your order...

Don't refresh or navigate away from the page.

liveProject

$35.99

liveProject liveProjects give you the opportunity to learn new skills by completing real-world challenges in your local development environment. Solve practical problems, write working code, and analyze real data—with liveProject, you learn by doing. These self-paced projects also come with full liveBook access to select books for 90 days plus permanent access to other select Manning products. $35.99 $59.99 you save: $24 (40%) self-paced learning

try it! first milestone FREE

Create a Data Platform for Real-time Anomaly Detection (liveProject) added to cart

continue shopping go to cart

continue shopping

go to cart

customers also bought these items

add to cart

The project has adopted modern technologies that are currently popular in the industry and academia. The tools and techniques used are very relevant.

Lawrence Nderu, Professor, Jomo Kenyatta University of Agriculture and Technology

Look inside

In this liveProject, you’ll close the gap between “data analyst” and “software engineer” by building a working data platform. You’ll join up with AnomalousDex Inc., a startup that specialises in personalized end-to-end data products, and create a working prototype of their anomaly detection platform to showcase to prospective customers. This requires connecting up multiple cross-discipline components, from data science to systems management.

This platform consists of three principal components: a service that serves the anomaly detection data model, the modelling platform, and a dashboard visualization tool. Your challenge is to develop all of these features, going hands-on with software architecture, data engineering, microservices, and dockerizing. You’ll also dive into the essentials of monitoring and metrics, and even train an unsupervised learning anomaly detection model!

This project is designed for learning purposes and is not a complete, production-ready application or solution.

book and video resources

When you start your liveProject, you get full access to the following books and videos for 90 days.

project author

Juan De Dios Santos

Juan is a Data Engineer at LOVOO, a dating and social platform. He's responsible for developing methods to detect and combat the fraudsters and spammers encountered in the platform. When he's not dealing with spam, you might find him experimenting with personal data to learn curious details about himself or playing with object detection models. You can find more about his work at https://juandes.com. Juan holds a BSc in Computer Science from the University of Puerto Rico - Rio Piedras Campus and an MSc in Computer Science from Uppsala University in Sweden and authored a book about TensorFlow.js.

prerequisites

This liveProject is for intermediate-level Python programmers. If your background is in data, you’ll learn how you can advance your isolated models to production. If you’re a developer, you’ll see how you can create and incorporate machine learning into your software products. To begin this liveProject you will need to be familiar with:

TOOLS

Basics of pandas
Basics of Matplotlib
Basics of scikit-learn
Basics of Docker

TECHNIQUES

Basics of exploratory data analysis
Basics of machine learning
Basics of software engineering

you will learn

In this liveProject, you’ll learn to use the Python data ecosystem, Docker, and monitoring tools like Prometheus and Grafana to create a complete end-to-end data platform. This kind of integrated data tool is in high demand across industries.

Design a data platform architecture consisting of several Dockerized components
Train an anomaly detection model with scikit-learn and deploy it in a web service
Analyze and visualize data in Jupyter
Monitor the web service and anomaly detection model with Prometheus and Grafana

features

Self-paced: You choose the schedule and decide how much time to invest as you build your project.
Project roadmap: Each project is divided into several achievable steps.
Get Help: While within the liveProject platform, get help from other participants and our expert mentors.
Compare with others: For each step, compare your deliverable to the solutions by the author and other participants.
book resources: Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.

I believe the course is great for the aspiring Data Engineer.

Pavel Filatov, Data Engineer, New Yorker