hands-on lab

Configure FTP service with S3 backend using S3FS

Difficulty: Intermediate

Duration: Up to 1 hour

Students: 322

Rating: 4.6/5

Start lab

Get guided in a real environmentPractice with a step-by-step scenario in a real, provisioned environment.

Learn and validateUse validations to check your solutions every step of the way.

See resultsTrack your knowledge and monitor your progress.

Description

Lab Overview

FTP (File Transfer Protocol) is a fast and convenient way to transfer large files over the Internet. You might, at some point, have configured an FTP server and used block storage, NAS, or a SAN as your backend. But using this kind of storage requires infrastructure support and can cost you a fair amount of both time and money.

Since November 2018 AWS has offered a managed SFTP service called AWS Transfer for SFTP. It offers the convenience of being highly available and automatically scaled. It also has built-in integration with other AWS services such as IAM and CloudWatch. If cost is not an issue, the managed service may be a good choice for you. This Lab explores a fully customizable, open source solution for using FTP to transfer files to an S3 bucket. The solution in this Lab also covers scenarios when you want to mount an S3 bucket on your local file system.

S3FS

In this lab, you will learn to download, compile, and install the S3FS Fuse utility onto an Amazon Linux 2 based EC2 instance. s3fs allows Linux based operating systems to mount an S3 bucket via FUSE. S3FS is an open source project maintained and located as a public git repository on GitHub. In this lab, you will download the s3fs source code, then compile and install the resulting s3fs executable, which will then be used to mount a Linux file system directory to an S3 bucket.

This lab is for anyone that is interested in configuring an FTP service to use Amazon's S3 service as its backend storage layer.

Lab Objectives

Upon completion of this lab, you will be able to:

Install and configure an FTP service
Download, compile and install the S3FS Fuse based executable
Configure and mount a Linux filesystem directory to an S3 bucket
Use both command line and desktop based FTP clients to upload and download files - which will, in turn, be transferred to and from a mounted S3 bucket

Lab Prerequisites

You should be familiar with:

Be comfortable with SSH to remotely administer a Linux based server
Basic Linux administration
Basic FTP commands

Lab Environment

This lab will start with the following AWS resources being provisioned automatically for you:

IAM Role (ec2-instance-role)
IAM Policy (s3-bucket-policy)
EC2 Amazon Linux 2 based instance - with ec2-instance-role attached
S3 Bucket

To achieve the lab end state, you will be walked through the process of:

Downloading, compiling, and installing the s3fs executable
Install and configure an FTP service
Mount a Linux directory using the s3fs command to the provided S3 bucket
Perform FTP testing from your local workstation - resulting in locally hosted files being FTP'd up to the remote EC2 instance. The s3fs process running on the EC2 instance will in turn propagate the files across to the configured S3 bucket.