dbt node level alerting
Setup
Metaplane leverages the meta
key of a dbt exposure
named metaplane_alerts to provide flexible and powerful alerting capabilities for your dbt models and tests.
Here's a simple top-level catch-all rule you can add to your schema.yml file:
exposures:
- name: metaplane_alerts # don't change, this is how we find the top level exposure
type: application
owner:
name: metaplane
meta:
metaplane:
alerting:
rules:
- name: All failures
description: catch all node level failures
includeWarnings: true # this defaults to true for all rule
destinations:
- type: slack_channel
name: data-engineering
- type: email
emailAddress: 'alerts@data_team.com'
This rule will send notifications for all model and test failures and warnings in a specific dbt run to both the specified Slack channel and the listed email address. To exclude a rule notifying for warnings, you can set includeWarnings: false
.
The alerting rules also allow you to target alerts based on node-level properties. For example, you can create a rule to notify only about failures related to "critical" models:
exposures:
- name: metaplane_alerts
type: application
owner:
name: metaplane
meta:
metaplane:
alerting:
rules:
- name: critical models
description: failures related to critical models
filters:
- type: TAG
tags:
- critical
- type: WAREHOUSE_LOCATION
schema: GOLD
destinations:
- type: slack_channel
name: high_priority
In this example, the rule will send alerts to the "high_priority" Slack channel whenever a model or test fails and the model is tagged as "critical" or lives in the "GOLD" schema.
Beyond top-level rules, you can also add alerting destinations directly to individual models:
models:
- name: stg_orders
meta:
metaplane:
alerting:
rules:
- name: source_test_sfdc
destinations:
- type: slack_channel
name: data-engineering
Specifying a node-level rule will send an alert to all the specified destinations if that node has any failures. If a model is tagged, any child tests will inherit the alert destinations.
You can have as many of these alerting rules as you want, and they can each be as broad or specific as you need. The available filters include:
- name: critical infra
description: failures related to critical infra
filters:
- type: TAG
tags:
- critical
- p0
- type: WAREHOUSE_LOCATION
database: ANALYTICS # these can be regex
schema: PROD
- type: OWNER
ownerName: todd
And the available alert destination types are:
destinations:
- type: slack_channel
name: data-engineering
- type: email
emailAddress: 'alerts@data_team.com'
Updated 10 days ago