Skip to content
hands-on lab

Using Azure Data Factory Pipelines to Copy Data

Difficulty: Intermediate
Duration: Up to 1 hour and 15 minutes
Students: 2,546
Rating: 4.5/5
Start lab
On average, students complete this lab in45m
Get guided in a real environmentPractice with a step-by-step scenario in a real, provisioned environment.
Learn and validateUse validations to check your solutions every step of the way.
See resultsTrack your knowledge and monitor your progress.
About
Author

Description

Azure Data Factory (ADF) is a managed cloud service for ingesting, preparing and transforming data from multiple sources. ADF provides code-free, visual data pipeline interface to describe workflows allowing data engineers and non-expert data integrators alike to accomplish complex data manipulation tasks. Over 90 data sources are supported including Azure services, Amazon Web Services, Google Cloud services, Teradata, and Salesforce.

This lab introduces you to Azure Data Factory. You will get acquainted with ADF by performing an Azure Blob Storage data movement operation using an ADF pipeline.

Warning: Currently, Data Factory UI is only officially supported by Microsoft Edge and Google Chrome web browsers. It is also recommended to use incognito mode to avoid conflicts with other Microsoft accounts impacting Data Factory UI.

Learning Objectives

Upon completion of this intermediate-level lab, you will be able to:

  • Create an Azure Data Factory
  • Understand and initialize ADF linked services
  • Initialize ADF datasets
  • Develop basic data pipelines in Data Factory UI
  • Learn to Debug and Trigger ADF pipelines
  • Triggers ADF pipeline runs on a schedule

Intended Audience

  • Candidates for Microsoft Azure Data Engineering Certifications
  • Data Engineers

Prerequisites

Familiarity with the following will be beneficial but is not required:

  • Basic understanding of Azure Data Factory
  • Basic understanding of Azure Blob Storage

The following courses can be used to fulfill the prerequisite:

Portions of this lab's content have been adapted from attributed sources.

Updates

October 3rd, 2023 - Resolved deployment issue

Environment before

Environment after

Covered topics

Microsoft Azure
Analytics for Azure
Azure Data Factory
Big Data for Azure
Big Data
Analytics
Hands-on Lab UUID

Lab steps

Logging in to the Microsoft Azure Portal
Creating an Azure Data Factory
Creating a linked service
Creating the Data Factory Datasets
Creating a Data Factory Pipeline
Debugging the Data Factory Pipeline
Triggering and Monitoring the Pipeline
Triggering the Pipeline on a Schedule

Lab Rules