5, 10 or 20 seats+ for your team - learn more
Welcome to Free Power Corporation Limited (FPCL), a London-based energy company looking for a solution to deal with surging energy costs. FPCL has installed Smart Meters, which generate energy readings every thirty minutes, in households across London. As a data engineer for FPCL, you’ll create a Kafka cluster and ingest the real-time Smart Meter data into it. You’ll use Spark to read, clean, join, and process the data, adding logic to handle potential real-world problems like data loss and duplicate data. To meet the different business requirements of various FPCL teams, you’ll also perform advanced stream processing on the data streams. By the end of this series of liveProjects, you’ll have the experience and skills to ingest large amounts of data and perform complex analysis on it in real time using Apache Kafka and Spark.
The project is taking a very good progressive way to bring the user from basics to advanced covering the foundations of Kafka.
It's a very good project to learn Spark Streaming with Kafka. Very well executed with simple steps.
For me, it is a definite game changer as I can now say I have real-life experience with event streaming as my current company (due to regulatory constraints) cannot adopt new technologies on a whim.
These liveProjects are for intermediate Scala developers and data engineers with basic knowledge of distributed computing technologies such as Apache Spark. To begin these liveProjects you’ll need to be familiar with the following:TOOLS
In this liveProject series, you’ll learn to use Kafka and Spark to ingest, stream, and process large amounts of data.
geekle is based on a wordle clone.