click to
look inside
Look inside

Create a Data Platform for Real-time Anomaly Detection

prerequisites
intermediate Python • basics of pandas • basics of Matplotlib • basics of scikit-learn • basics of Docker • basics of exploratory data analysis • basics of machine learning • basics of software engineering
skills learned
design a data platform architecture consisting of several Dockerized components • train an anomaly detection model with scikit-learn and deploy it in a web service • monitor the web service and anomaly detection model with Prometheus and Grafana
Juan De Dios Santos Rivera
7 weeks · 5-7 hours per week · INTERMEDIATE

placing your order...

Don't refresh or navigate away from the page.
liveProject liveProjects give you the opportunity to learn new skills by completing real-world challenges in your local development environment. These self-paced projects also come with full liveBook access to select books for 90 days plus permanent access to other select Manning products. $35.99 $59.99 you save: $24 (40%) self-paced learning
FREE domestic shipping on orders of three or more print books
Create a Data Platform for Real-time Anomaly Detection (liveProject) added to cart
continue shopping
go to cart

The project has adopted modern technologies that are currently popular in the industry and academia. The tools and techniques used are very relevant.

Lawrence Nderu, Professor, Jomo Kenyatta University of Agriculture and Technology
Look inside
In this liveProject, you’ll close the gap between “data analyst” and “software engineer” by building a working data platform. You’ll join up with AnomalousDex Inc., a startup that specialises in personalized end-to-end data products, and create a working prototype of their anomaly detection platform to showcase to prospective customers. This requires connecting up multiple cross-discipline components, from data science to systems management.

This platform consists of three principal components: a service that serves the anomaly detection data model, the modelling platform, and a dashboard visualization tool. Your challenge is to develop all of these features, going hands-on with software architecture, data engineering, microservices, and dockerizing. You’ll also dive into the essentials of monitoring and metrics, and even train an unsupervised learning anomaly detection model!
This project is designed for learning purposes and is not a complete, production-ready application or solution.

book and video resources

When you start your liveProject, you get full access to the following books and videos for 90 days.

project author

Juan De Dios Santos
Juan is a Data Engineer at LOVOO, a dating and social platform. He's responsible for developing methods to detect and combat the fraudsters and spammers encountered in the platform. When he's not dealing with spam, you might find him experimenting with personal data to learn curious details about himself or playing with object detection models. You can find more about his work at https://juandes.com. Juan holds a BSc in Computer Science from the University of Puerto Rico - Rio Piedras Campus and an MSc in Computer Science from Uppsala University in Sweden and authored a book about TensorFlow.js.

prerequisites

This liveProject is for intermediate-level Python programmers. If your background is in data, you’ll learn how you can advance your isolated models to production. If you’re a developer, you’ll see how you can create and incorporate machine learning into your software products. To begin this liveProject you will need to be familiar with:

TOOLS
  • Basics of pandas
  • Basics of Matplotlib
  • Basics of scikit-learn
  • Basics of Docker
TECHNIQUES
  • Basics of exploratory data analysis
  • Basics of machine learning
  • Basics of software engineering

you will learn

In this liveProject, you’ll learn to use the Python data ecosystem, Docker, and monitoring tools like Prometheus and Grafana to create a complete end-to-end data platform. This kind of integrated data tool is in high demand across industries.

  • Design a data platform architecture consisting of several Dockerized components
  • Train an anomaly detection model with scikit-learn and deploy it in a web service
  • Analyze and visualize data in Jupyter
  • Monitor the web service and anomaly detection model with Prometheus and Grafana

features

Self-paced
You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from other participants and our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.

I believe the course is great for the aspiring Data Engineer.

Pavel Filatov, Data Engineer, New Yorker
RECENTLY VIEWED