Often, data engineers can't know the full downstream impact of changes made to dbt model code. Metaplane's CI/CD tools automatically predict the downstream impact of dbt model changes (Data Impact Previews) and run a comprehensive suite of tests to help you understand how values in those downstream tables will change (Data Test Previews), all before you merge a single code change.

Teams use Metaplane's CI/CD tool to:

Avoid making breaking changes to their dbt models and downstream dependencies.
More quickly understand the root cause of errant code changes when they occur.
Highlight the importance of data-quality right in their source-control tools.

Features

Impact Previews

Impact previews show the downstream linage that could potentially be affected by your dbt changes. Metaplane will leave comments like this on your pull requests:

Test Previews

Test previews run a battery of automated tests against your dbt changes, highlighting changes to tables and columns. Metaplane will leave comments like this on your pull requests:

Dashboard

In addition to comments left on your pull requests, there's a purpose-built CI/CD Dashboard in Metaplane. You can explore past pull requests and run results in a single place:

Setup

Connect Your Provider

Following the instructions in-app, connect your git provider to Metaplane. For more detailed instructions, select the provider you want to connect below.

Configure CI/CD

Once you've connected your git provider, we'll need a URL for your production dbt job. You can optionally provide a set of allowed file locations for more fine-tuned control over which models Metaplane will report on.

Open the settings from the CI/CD dashboard

Provide the URL of your production dbt job.

Impact Previews

To configure impact previews, open the settings from the CI/CD dashboard.

Use the toggle to turn on impact previews and configure.

Optional settings include:

Ignore draft pull requests

Test Previews

To create test previews, Metaplane will need access to the PR schemas that your dbt CI job generates on each run. You can give us access in one of two ways:

Grant the Metaplane user access to all future schemas on the specific warehouse database where dbt is running.
Add a step in the CI job in dbt to selectively grant the Metaplane user access to the newly created schema. Our friends at SpotOn created this helpful macro to grant access to the newly created schemas. This can be adapted and run in your project via dbt run-operation command.

Once you've set up the appropriate permissions, open the settings from the CI/CD dashboard:

Use the toggle to turn on test previews and configure the settings you want in-app.

A few fields are required:

Target Warehouse - which data warehouse you’d like your regression tests run against
CI Job URL - The job that kicks off your CI workflow; note that this is different from the production dbt job URL you configured previously.

Optional settings include:

Ignore draft pull requests
Test downstream dbt models
Tag filters - Using this option will restrict your tests only to specific Metaplane tags that you’ve applied.
CI Job timeout
Query time
Change threshold - Configure what percent deviation in a value constitutes a failure.