AWS Big Data Specialty - Data Collection

In lesson one of the AWS Big Data Specialty Data Collection course we explain the various data collection methods and techniques for determining the operational characteristics of a collection system. We explore how to define a collection system able to handle the frequency of data change and the type of data being ingested. We identify how to enforce data properties such as order, data structure, and metadata, and to ensure the durability and availability for our collection approach.

Learning Objectives

• Recognize and explain the operational characteristics of a collection system.
• Recognize and explain how a collection system can be designed to handle the frequency of data change and the type of data being ingested.
• Recognize and identify properties that may need to be enforced by a collection system.

Intended Audience

This lesson is intended for students looking to increase their knowledge of data collection methods and techniques with big data solutions.

Prerequisites

While there are no formal prerequisites, students will benefit from having a basic understanding of analytics services available in AWS. Please take a look at our Analytics Fundamentals for AWS

This Lesson Includes

• 45 minutes of high-definition videos
• Live hands-on demos

What You'll Learn

• Introduction to Collecting Data: In this lesson, we'll prepare you for what we'll be covering in the lesson; the Big Data collection services of AWS Data Pipeline, Amazon Kinesis, and AWS Snowball.
• Introduction to Data Pipeline: In this lesson, we'll discuss the basics of Data Pipeline.
• AWS Data Pipeline Architecture: In this lesson, we'll go into more detail about the architecture that underpins the AWS Data Pipeline Big Data Service.
• AWS Data Pipeline Core Concepts: In this lesson, we'll discuss how we define data nodes, access, activities, schedules, and resources.
• AWS Data Pipeline Reference Architecture: In this lesson, we'll look at a real-life scenario of how data pipeline can be used.
• Introduction to AWS Kinesis: In this lesson, we'll take a top-level view of Kinesis and its uses.
• Kinesis Streams Architecture: In this lesson, we'll look at the architecture that underpins Kinesis.
• Kinesis Streams Core Concepts: In this lesson, we'll dig deeper into the data records.
• Kinesis Streams Firehose Architecture: In this lesson, we'll look at firehose architecture and the differences between it and Amazon Kinesis Streams.
• Firehose Core Concepts: Let's take a deeper look at some details about the Firehose service.
• Kinesis Wrap-Up: In this summary, we'll look at the differences between Kinesis and Firehose.
• Introduction to Snowball: Overview of the Snowball Service.
• Snowball Architecture: Let's have a look at the architecture that underpins the AWS Snowball big data service
• Snowball Core Concepts: In this lesson, we'll look at the details of how Snowball is engineered to support data transfer.
• Snowball Wrap-Up: A brief summary of Snowball and our lesson.