Monitoring Data Quality Issues with Amazon SageMaker

Amazon SageMaker is a service that enables you to build, train, and deploy machine learning models in the public AWS cloud. Monitoring deployed models over time helps you identify data quality issues and data drift that can affect model predictions.

Amazon SageMaker Model Monitor is no longer available to new AWS customers. In this lab, you will use the open-source Evidently AI library, referenced in AWS replacement guidance, to generate a data quality and drift monitoring report for a deployed SageMaker endpoint.

In this hands-on lab, you will use a Jupyter notebook to examine synthetic datasets, inspect a deployed SageMaker endpoint and its data capture configuration, review baseline artifacts, and generate an Evidently AI monitoring report.

Please note: This lab uses an Amazon SageMaker notebook and endpoint, which can take up to ten minutes to deploy. Please ensure you have enough time available before starting the lab.

Learning objectives

Upon completion of this beginner-level lab, you will be able to:

• Access a JupyterLab notebook
• Generate and examine a synthetic dataset
• Examine a deployed model endpoint
• Inspect endpoint data capture and baseline artifacts
• Generate a data quality and drift monitoring report with Evidently AI

Intended audience

• Candidates for the AWS Certified Machine Learning Engineer Associate certification
• Cloud Architects
• Data Engineers
• DevOps Engineers
• Machine Learning Engineers

Prerequisites

Familiarity with the following will be beneficial but is not required:

• Amazon SageMaker
• Amazon S3
• The Python programming language

The following content can be used to fulfill the prerequisites:

• Amazon SageMaker: Machine Learning Workflows
• Introduction to Amazon S3
• Python for Beginners

Updates

Sep 1st, 2026 - Updated notebook data quality monitoring flow