Working With Incremental dbt Models

If you are working with models that handle recorded event data or facts, you could fall into a situation where your models are completely refreshed at every dbt execution and most of the data was already in place in your tables. This may be because these types of data rarely change once created (e.g bank transactions).

Incremental dbt models work well in these scenarios and help speed up the dbt model creation and not overhead databases. Incremental models are materialized as tables, but they are not fully refreshed at every dbt execution. Only new records that were not previously in the table are added.

In this lab step, you will create two dbt incremental models by using two different strategies in order to let dbt understand whether a row needs to be added or not.

Learning Objectives

Upon completion of this lab, you will be able to:

• Understand what are incremental models
• Work and create incremental dbt models

Intended Audience

• Data engineers with a focus on data modeling and transformations methodologies
• Developers that need to understand different kinds of dbt materialization techniques

Prerequisites

To get the most from this lab, you should have basic knowledge of dbt. To get ready, you can use the following labs:

• Create Your First dbt (Data Build Tool) Project
• Configure a dbt Profile and Define Sources
• Create and Execute Your First dbt Models
• Working With Full-Refresh dbt Models