dbt is a data transformation tool that enables data analysts and engineers to transform, test and document data in the cloud data warehouse.
Metaplane can access your dbt cloud metadata to extract metadata about job runs metadata such as run durations, models generated by dbt, and lineage relationships between models. Metaplane is able to match tables in your warehouse with dbt models to determine the causality and consequences of table failure.
Paid dbt plans only
The dbt Cloud API is only available to Team and Enterprise customers.
Enabling API access
Navigate to dbt Cloud: https://cloud.getdbt.com
Click on the top left hamburger menu button > Account Settings
- Click the button Enable Metadata Access
Retrieving Service Account Token
- Click on your profile icon > Account Settings
- Click Service Tokens. Then click New Token
- Enter a token name, like Metaplane. Then click + Add and select the Job Admin permission set for All Projects. Lastly, click Save in the bottom right hand corner to create the service token.
- Once you click Save, you will be provided a service token. This is what Metaplane will use to start monitoring your dbt projects.
Add Service Token to Metaplane
- Navigate to https://app.metaplane.dev/connections
- Click on the Add connection button
- Select the dbt card
- Add the dbt API token into the modal and save
The historical run time data for dbt jobs will immediately start syncing. You can see these jobs by clicking on the "jobs" navigation tab, or going to https://app.metaplane.dev/jobs.
Downstream/upstream lineage from dbt will be available within tests and alerts within 24 hours.
GitHub Integration (Beta)
Level up your data team's insight into how pull requests will affect your data stack before they're even merged. Metaplane's GitHub App will enrich your team's pull requests with helpful context ranging from downstream model and BI impact to data quality implications on your production warehouse.
Using dbt and BI lineage, our Github bot will comment on each of your dbt PRs and report potential downstream impacts.
Get notified of how pull requests will impact the data quality of your warehouse. Metaplane will compare the results of running a suite of data tests against your PR changes and production warehouse.
- Navigate to your dbt connection either via the Connections page or through the sidebar link.
- Click the "Connect to GitHub" button in the upper right-hand corner of the screen. The app will then take you through a guided connection flow. We recommend you apply our GitHub App only to your specific dbt repo.
- Once the Metaplane GitHub App has been successfully installed, you'll get redirect back to the Metaplane app where we'll be prompted to complete the setup process. Click "Setup dbt integration."
If installing the Metaplane GitHub app required you to request GitHub admin approval, please reach out to Metaplane support once the app has been approved so that we can help you complete the process.
- Complete the form by selecting the GitHub App features that you'd like to enable along with the required dbt job information.
Once you're finished with the steps above there is nothing left for you to do. The GitHub Metaplane App will automatically respond to pull requests with related context, per the features that you've enabled.
Updated about 2 months ago