hands-on lab

Aggregating Data with Amazon Managed Streaming for Apache Kafka (MSK)

Difficulty: Beginner

Duration: Up to 2 hours

Students: 1,467

Rating: 4.3/5

Start lab

Get guided in a real environmentPractice with a step-by-step scenario in a real, provisioned environment.

Learn and validateUse validations to check your solutions every step of the way.

See resultsTrack your knowledge and monitor your progress.

Description

Amazon Managed Streaming for Apache Kafka (also known as Amazon MSK) is an event streaming platform that's capable of handling events numbering in the trillions per day. Originally Apache Kafka was designed to be a type of message queue, it has proven itself useful in many other use-cases too.

This managed offering from AWS makes reliably setting up and managing Apache Kafka clusters simple. You don't need to worry about provisioning servers, or keeping them patched up to date. Amazon MSK integrates with existing AWS technology. Storage is secure and durable, and monitoring is taken care of with Amazon CloudWatch.

In this Hands-On lab, you will see how to create a cluster configuration for an Amazon MSK cluster. You will connect to an Amazon MSK cluster and create some Topics. And you will create a simple application using the Faust streaming library that populates the Topics and aggregates the data.

Please note, this lab creates an Amazon MSK cluster which can take over twenty minutes to finish setting up. Please make sure you have enough time available before starting this lab.

Learning Objectives

Upon completion of this beginner-level lab, you will be able to:

Create an Amazon MSK Cluster Configuration
Create a Topic in an Amazon MSK cluster using the Apache Kafka command-line tools
Implement a Python script that aggregates Topic data
Retrieve data from Topics using the command-line and Python