Processing Streaming Metadata using Amazon Kinesis Data Streams

Amazon Kinesis Data Streams (KDS) is a serverless data streaming service available in the public AWS cloud. It's a fully managed service that can be used to store and ingest streaming data in real time at scale. Amazon KDS integrates with other AWS services as well as being easy to use from custom applications.

Learning how to set up, configure, and use Amazon KDS will enable you to build systems in the AWS cloud that can handle high throughput data rates.

In this hands-on lab, you will create a new Amazon Kinesis data stream and configure an AWS Lambda function to process data records from the stream.

Learning Objectives

Upon completion of this beginner level lab, you will be able to:

• Create a new Amazon Kinesis Data Stream
• Configure an AWS Lambda function to process data stream records
• Configure an Amazon Simple Queue Service (SQS) queue to handle processing failures

Intended Audience

• Candidates for the AWS Certified Data Analytics Specialty certification
• Cloud Architects
• Data Engineers
• DevOps Engineers
• Software Engineers

Prerequisites

Familiarity with the following will be beneficial but is not required:

• Amazon Kinesis Data Streams (KDS)
• AWS Lambda
• Amazon Simple Queue Service (SQS)

The following content can be used to fulfill the prerequisites:

• Amazon Kinesis Data Streams Part 1
• Amazon Kinesis Data Streams Part 2
• Understanding AWS Lambda to Run & Scale Your Code
• Decoupled and Event-Driven Architectures in AWS

Updates

February 22nd, 2024 - Resolved deployment issue

October 25th, 2023 - Updated the instructions and screenshots to reflect the latest UI