With test previews enabled, Metaplane will run a battery of automated tests against pull requests, highlighting any significant changes in values. These reports will help prevent breaking changes from being merged, as well as tell you precisely how your changes will affect the data.

Setup

To create test previews, Metaplane will need access to the PR schemas that your dbt CI job generates on each run. You can give us access in one of two ways:

1. Grant the Metaplane user access to all future schemas on the specific warehouse database where dbt is running.
2. Add a step in the CI job in dbt to selectively grant the Metaplane user access to the newly created schema. Our friends at SpotOn created this helpful macro to grant access to the newly created schemas. This can be adapted and run in your project via dbt run-operation command.

Once you've set up the appropriate permissions, open the settings from the CI/CD dashboard:

Use the toggle to turn on test previews and configure the settings you want in-app.

A few fields are required:

• Target Warehouse - which data warehouse you’d like your regression tests run against
• CI Job URL - The job that kicks off your CI workflow; note that this is different from the production dbt job URL you configured previously.

Optional settings include:

• Ignore draft pull requests
• Test downstream dbt models
• Tag filters - Using this option will restrict your tests only to specific Metaplane tags that you’ve applied.
• CI Job timeout
• Query time
• Change threshold - Configure what percent deviation in a value constitutes a failure.

Advanced configuration

Gain more control over how Metaplane runs your test by specifying additional configuration in your dbt model's metadata section. Below is an annotated example of the currently supported options.

- name: payments
  meta:
    metaplane:
      testPreviews:
        # optional custom sql expression(s) to limit the 
        # data tested on both staging and production version of this model
        sqlWhereClause: "payment_id > 20 AND created_at > dateadd('day', -5, current_timestamp)"

        # optional, defaults to true. Set to false to turn off test previews for this model
        isEnabled: true

sqlWhereClause - This flag limits what data Metaplane looks at. You may want to use this if you're looking to:

• Improve test runtime on large tables
• Ensure that the range of data being queried matches on both staging and production

Troubleshooting

Unable to find Tables and Columns

This is one of most common reason we are not able to run tests. Metaplane attempts to find the table + column for each changed model both from the normal warehouse path as well as in the newly created dbt_pr schema. This error occurs when Metaplane can't find a table on the dbt_pr schema.

To resolve this issue:

• Ensure Metaplane has been granted read permissions for the dbt_pr schemas (see Setup above)
• Automation like SlimCI will prevent unchanged models from being built in each PR. This can prevent the model from existing on the dbt_pr schema even when we have access.
• A net new model doesn't yet have a comparison point, so we can't test it.
• If none of the above are true, and the model appears in both the warehouse and the dbt_pr schema, please reach out for support.