Gianluigi Mucciolo

Nexstellar Corporation is one of the leading messaging and multimedia service providers in Italy, and they want to make their customer experience even better! They’ve recruited you to spearhead an ambitious project to implement a Lambda architecture on AWS. Over four interconnected projects, you'll build a state-of-the-art real-time data processing pipeline that transforms how modern businesses handle and leverage massive data streams. Starting with data ingestion using Amazon Kinesis, you'll progress through increasingly complex challenges: creating a robust data lake, implementing streaming analytics with Apache Flink, and finally developing an intelligent system using OpenSearch and Amazon Bedrock. You'll master cutting-edge cloud technologies like AWS Glue, Apache Zeppelin, and serverless architectures while learning to turn raw data into actionable insights that drive business decisions

This series uses Amazon Web Services, which should cost less than 18 USD for the whole series, including cleanup.

In this liveProject, you'll transform Nexstellar Corp's data architecture by building a sophisticated OpenSearch solution powered by Amazon Bedrock. You'll extract embedding features from logs, create a dynamic Knowledge Base, and develop an intelligent agent with robust generative AI guardrails. Your challenge: design a flexible database system that can efficiently store, query, and analyze real-time data while seamlessly integrating with existing infrastructure. By project's end, you'll have the skills to turn complex data streams into actionable insights that drive business innovation!

This project uses Amazon Web Services, which should cost less than 5 USD for the whole project, including cleanup.

In this liveProject, you'll harness the power of Apache Flink to build a real-time data transformation pipeline for Nexstellar Corp. You'll master reading data from Amazon Kinesis using an enhanced fan-out configuration, write SQL queries in Apache Zeppelin, and create real-time visualizations. Your challenge: Deploy a fully operational streaming analytics application on AWS, starting from a Zeppelin notebook, and conduct a comprehensive comparison between AWS Glue Streaming and Apache Flink.

This project uses Amazon Web Services, which should cost less than 7 USD for the whole project, including cleanup.

In this liveProject, you'll transform raw streaming data into a powerful, integrated data lake using cutting-edge tools like Apache Spark Structured Streaming, AWS Glue, and Amazon Athena. You'll develop a robust system that can process streaming information, update existing datasets, and enable lightning-fast analytics that give your company a competitive edge in the fast-moving multimedia market.

This project uses Amazon Web Services, which should cost less than 3 USD for the whole project, including cleanup.

In this liveProject, you'll step into the role of a data engineering specialist at Nexstellar Corp to tackle the challenge of transforming the company's data processing capabilities. As Nexstellar seeks to evolve from batch to real-time data analysis, you'll develop a streaming data pipeline using Amazon Kinesis and AWS Lambda. You'll design a system that can rapidly ingest, transform, and store streaming data, enabling the multimedia service provider to make lightning-fast, data-driven decisions. By the end of this project, you'll have constructed a robust, scalable streaming solution that turns raw data into actionable insights!

This project uses Amazon Web Services, which should cost less than 2 USD for the whole project, including cleanup.

Nextstellar Corp is a streaming media company that generates huge amounts of data from its customers. They want to move their data infrastructure from on-prem solutions that analyze only a sample of data to a modern cloud solution that gives them full access to all the data they produce. That’s where you come in! In this series of liveProjects, you’ll build an Extract, Transform, and Load (ETL) solution that can transfer data from numerous existing sources to the AWS cloud. You’ll learn how to code raw data transformation logic; use AWS Glue jobs to normalize, transform, and validate data quality rules; coordinate jobs into seamless workflows with AWS Step Functions; and more.

Nextstellar Corp needs you to tackle a big challenge: completing their cloud migration by rebuilding their historical data lake as a data layer in the cloud. You’ll implement an effective and automated data orchestration framework for the ingestion and transform and curate layers, using best practices for Infrastructure-as-Code to automate your data layer. Finally, you’ll establish a monitoring system that will automatically alert you to any issues or problems that might crop up.

Nextstellar Corp has recently migrated to the cloud, and for the first time, they can analyze 100% of their company’s data. But there’s a problem: your CEO isn’t confident in your data’s quality. He wants to add more data sources and collect more user behavior information, and ensure these new sources are top-notch by utilizing the Python- (or Scala-) based Deequ library. Your task is to utilize Jupyter Notebooks with AWS Glue Studio to experiment with PyDeequ for data quality assessment. Next, you’ll enhance Nextstellar’s assessment capabilities by employing AWS Glue jobs to react and take action on data quality issues. Finally, you’ll monitor data quality using CloudWatch for prompt response and maintenance of data reliability.

Nextstellar Corp is very excited—they’re taking all their data to the cloud! They’ve recently migrated and have an early-defined data lake solution. The CEO has approached you to deliver the next step of their cloud data process: using AWS Glue to apply the transformation logic and store the curated data in Amazon S3. You’ll utilize Jupyter Notebooks to curate and standardize your data sets—crafting, organizing, and managing datasets to ensure they are easily accessible and usable—then design a CI/CD pipeline to test and deploy code with a single push after completion.

Media company Nextstellar Corp has completed the migration of their data to the cloud—now they need to analyze it! That’s where you come in. You’ll take on the challenge of processing and integrating file and transactional data into a new cloud database solution that uses modern data architecture. You’ll use the AWS Glue tool to automate the whole process, from creating your crawlers and database to building your repository, Glue jobs, triggers, and establishing monitoring, troubleshooting, and scaling.

Nextstellar Corp is a media company with huge amounts of data to analyze. Some of that data is sitting in a PostgreSQL database, which is used for both authentication management and decision-making, as well as maintaining user preferences and feedback. Your boss doesn’t want that data sitting in the database—he wants it in the cloud! Moving it is exactly what you’ll be doing in this liveProject. You’ll use AWS Database Migration Service to enrich Nextstellar’s data lake with the PostgreSQL database so it can take full advantage of both modern data architecture and the AWS ecosystem.

The Nextstellar Corp media service has a lot of data—too much data to handle on prem! In order to properly analyze all the data generated by their streaming customers, they’re migrating to the cloud. In this liveProject, you’ll be helping them. You’ll tackle the common challenge of transferring on-prem data to AWS using the handy AWS DataSync tool. You’ll use Infrastructure-as-Code to create Landing Zone Amazon S3 buckets, automate data migration, and finally prepare a summary of likely infrastructure costs for your boss to review.