Monitor types

Metaplane supports the following types of monitors out-of-the-box, from monitoring row counts and freshness to mean and standard deviation.

Row Count

Description

Row count monitors track the number of rows in a table or view over time. They are great for ensuring regularly updated tables don’t have any issue. Our anomaly detection will automatically alert you the row count is lower or higher than expected.

Details

When able, Metaplane will pull the row count data for tables and materialized views directly from the information_schema tables, making these much cheaper monitors to run. Metaplane is able to automatically for all warehouse connections except Postgres, SQLServer, and MySQL.

If the information_schema can’t serve the query, we will run a sql query like: SELECT COUNT(*) FROM TABLE

Freshness

Description

Freshness tracks the duration since a table(limited to specific warehouses) or column was last updated. Our anomaly detection attempts to learn when your table or column should be updated and alerts if it has been too long since the table or column has been updated.

Details

When able, Metaplane will pull a last modified date for tables and materialized views directly from the information_schema tables, making these much cheaper monitors to run. Metaplane is only able to pull information_schema last modified dates for Snowflake, BigQuery, MySQL, and Databricks.

If freshness monitor is run on a column we will run sql query like: SELECT MAX(<yourColumn>) FROM TABLE to determine the last update time.

Column Count

Description

The Column count monitor tracks the number of columns in a given table or view. It will alert if the number of columns is either higher or lower than expected.

Details

Column Counts are gathered directly by querying the information schema.

Cardinality

Description

Cardinality checks the number of distinct elements of a given column. It will alert if the count if distinct elements is higher or lower than expected. The cardinality monitor is a great for checking enum like columns.

Details

Cardinality is run via sql like: SELECT COUNT(DISTINCT <yourColumn>) FROM TABLE

Uniqueness

Description

Uniqueness tracks the percentage of unique elements in a given a column. A value of 1 would mean the column is 100% unique while a value of 0 would mean the column is 0% unique (e.g. all the same value). The monitor will alert if the value is higher or lower than expected. This monitor is good for validating primary key columns.

Details

Uniqueness is run via sql like: SELECT COUNT(DISTINCT <yourColumn>) / COUNT(1) FROM TABLE

Nullness

Description

Nullness tracks the percentage of null elements in a given a column. A value of 0 would mean the column is 100% non-null while a value of 1 would mean the column is all null. The monitor will alert if the value is higher or lower than expected.

Details

Nullness is run via sql like: SELECT SUM(CASE WHEN <yourColumn> IS NULL THEN 1 ELSE 0 END) / COUNT(1) FROM TABLE

Minimum

Description

Minimum tracks the lowest value of a numeric column. It will alert when the value is higher or lower than expected.

Details

Minimum is run via sql like: SELECT MIN(<yourColumn>) FROM TABLE

Maximum

Description

Maximum tracks the highest value of a numeric column. It will alert when the value is higher or lower than expected.

Details

Maximum is run via sql like: SELECT MAX(<yourColumn>) FROM TABLE

Mean

Description

Mean tracks the arithmetic mean of a numeric column. It will alert when the value is higher or lower than expected.

Details

Mean is run via sql like: SELECT AVG(<yourColumn>) FROM TABLE

Standard Deviation

Description

The standard deviation monitor tracks standard deviation numeric column. It will alert when the value is higher or lower than expected.

Details

Standard Deviation is run via sql like: SELECT STDDEV(<yourColumn>::double) FROM TABLE

Sum

Description

The sum monitor tracks the sum of a numeric column. It will alert when the value is higher or lower than expected.

Details

Sum is run via sql like: SELECT SUM(<yourColumn>) FROM TABLE

Percent Zero

Description

Percent Zero tracks the percentage of values equal to zero in a given numeric column. A value of 0 would mean the column is 100% zero while a value of 1 would mean the column has no zeroes. The monitor will alert if the value is higher or lower than expected.

Details

Percent Zero is run via sql like: SELECT SUM(CASE WHEN <yourColumn> = 0 THEN 1 ELSE 0 END) / COUNT(1) FROM TABLE

Percent Negative

Description

Percent negative tracks the percentage of values less than zero in a given numeric column. A value of 0 would mean the column is 100% negative while a value of 1 would mean the column is all positive or zero. The monitor will alert if the value is higher or lower than expected.

Details

Percent Negative is run via sql like: SELECT SUM(CASE WHEN <yourColumn> < 0 THEN 1 ELSE 0 END) / COUNT(1) FROM TABLE

Custom SQL

Description

The custom SQL monitor allows you to enter your own totally custom sql query, giving you the ability to create much more complex and specify monitors.

Details

If your query returns a scalar numeric value (e.g. 50), Metaplane tracks that value over time and alerts you if the value is higher or lower than expected. If your query returns a result set, Metaplane tracks the number of rows returned over time and alerts if the value is higher or lower than expected.