Exporting Amazon CloudWatch Logs to Amazon S3 Using Kinesis

Amazon Kinesis Data Firehose is a streaming Extract, Transform, and Load (ETL) service from AWS. It's fully-managed, and once configured it scales to match your data throughput with no on-going administration.

Combining Amazon Kinesis Data Firehose with Amazon CloudWatch Logs and Amazon S3 allows you to build a solution that is capable of centralizing logs across many AWS accounts.

This can have the following benefits for your organization:

• Make long term data management easier
• Help to secure your data
• Help to meet regulatory or compliance requirements

In this lab, you will create a Kinesis Data Firehose Delivery Stream and use it to centralize logs from Amazon CloudWatch.

Learning Objectives

Upon completion of this beginner level lab, you will be able to:

• Create a Firehose Delivery Stream
• Subscribe a CloudWatch log group to your delivery stream
• Modify an Amazon S3 bucket to it more secure

Intended Audience

• Candidates for the AWS Certified DevOps Engineer - Professional certification
• DevOps Engineers
• Cloud Architects
• Data Engineers
• Software Engineers

Prerequisites

Familiarity with the following will be beneficial but is not required:

• Kinesis Data Firehose
• Amazon CloudWatch Logs
• Amazon Simple Storage Service (S3)

The following content can be used to fulfill the prerequisite:

• Overview of Amazon Kinesis
• Access Real-Time Log Events using CloudWatch Log Subscriptions
• Increasing Your Security Posture when Using Amazon S3