Create a Data Platform for Real-Time Anomaly Detection you own this product

prerequisites
intermediate Python • basics of pandas • basics of Matplotlib • basics of scikit-learn • basics of Docker • basics of exploratory data analysis • basics of machine learning • basics of software engineering
skills learned
design a data platform architecture consisting of several Dockerized components • train an anomaly detection model with scikit-learn and deploy it in a web service • monitor the web service and anomaly detection model with Prometheus and Grafana
Juan De Dios Santos Rivera
7 weeks · 5-7 hours per week · INTERMEDIATE
Included with a Manning Online subscription
try now

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

The project has adopted modern technologies that are currently popular in the industry and academia. The tools and techniques used are very relevant.

Lawrence Nderu, Professor, Jomo Kenyatta University of Agriculture and Technology
Look inside

In this liveProject, you’ll close the gap between “data analyst” and “software engineer” by building a working data platform. You’ll join up with AnomalousDex Inc., a startup that specializes in personalized end-to-end data products, and create a working prototype of their anomaly detection platform to showcase to prospective customers. This requires connecting up multiple cross-discipline components, from data science to systems management.

This platform consists of three principal components: a service that serves the anomaly detection data model, the modeling platform, and a dashboard visualization tool. Your challenge is to develop all of these features, going hands-on with software architecture, data engineering, microservices, and dockerizing. You’ll also dive into the essentials of monitoring and metrics, and even train an unsupervised learning anomaly detection model!

This project is designed for learning purposes and is not a complete, production-ready application or solution.
Manning author Juan De Dios Santos Rivera shares what he likes about the Manning liveProject platform.

book and video resources

When you start your liveProject, you get full access to the following books and videos for 90 days.

I believe the course is great for the aspiring Data Engineer.

Pavel Filatov, Data Engineer, New Yorker

project author

Juan De Dios Santos
Juan is a Data Engineer at LOVOO, a dating and social platform. He's responsible for developing methods to detect and combat the fraudsters and spammers encountered in the platform. When he's not dealing with spam, you might find him experimenting with personal data to learn curious details about himself or playing with object detection models. You can find more about his work at https://juandes.com. Juan holds a BSc in Computer Science from the University of Puerto Rico - Rio Piedras Campus and an MSc in Computer Science from Uppsala University in Sweden and authored a book about TensorFlow.js.

prerequisites

This liveProject is for intermediate-level Python programmers. If your background is in data, you’ll learn how you can advance your isolated models to production. If you’re a developer, you’ll see how you can create and incorporate machine learning into your software products. To begin this liveProject, you will need to be familiar with the following:

TOOLS
  • Basics of pandas
  • Basics of Matplotlib
  • Basics of scikit-learn
  • Basics of Docker
TECHNIQUES
  • Basics of exploratory data analysis
  • Basics of machine learning
  • Basics of software engineering

you will learn

In this liveProject, you’ll learn to use the Python data ecosystem, Docker, and monitoring tools like Prometheus and Grafana to create a complete end-to-end data platform. This kind of integrated data tool is in high demand across industries.

  • Design a data platform architecture consisting of several Dockerized components
  • Train an anomaly detection model with scikit-learn and deploy it in a web service
  • Analyze and visualize data in Jupyter
  • Monitor the web service and anomaly detection model with Prometheus and Grafana

features

Self-paced
You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from other participants and our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.

choose your plan

pro

monthly
annual
$24.99
$249.99
only $20.83 per month
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • Create a Data Platform for Real-Time Anomaly Detection project for free

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • Create a Data Platform for Real-Time Anomaly Detection project for free