Monitor types
Metaplane supports the following types of monitors out-of-the-box, from monitoring row counts and freshness to mean and standard deviation.
Row Count
Description
Row count monitors track the number of rows in a table or view over time. They are great for ensuring regularly updated tables don’t have any issue. Our anomaly detection will automatically alert you the row count is lower or higher than expected.
Details
When able, Metaplane will pull the row count data for tables and materialized views directly from the information_schema tables, making these much cheaper monitors to run. Metaplane is able to automatically for all warehouse connections except Postgres, SQLServer, and MySQL.
If the information_schema can’t serve the query, we will run a sql query like: SELECT COUNT(*) FROM TABLE
Freshness
Description
Freshness tracks the duration since a table(limited to specific warehouses) or column was last updated. Our anomaly detection attempts to learn when your table or column should be updated and alerts if it has been too long since the table or column has been updated.
Details
When able, Metaplane will pull a last modified date for tables and materialized views directly from the information_schema tables, making these much cheaper monitors to run. Metaplane is only able to pull information_schema last modified dates for Snowflake, BigQuery, MySQL, and Databricks.
If freshness monitor is run on a column we will run sql query like: SELECT MAX(<yourColumn>) FROM TABLE to determine the last update time.
Column Count
Description
The Column count monitor tracks the number of columns in a given table or view. It will alert if the number of columns is either higher or lower than expected.
Details
Column Counts are gathered directly by querying the information schema.
Cardinality
Description
Cardinality checks the number of distinct elements of a given column. It will alert if the count if distinct elements is higher or lower than expected. The cardinality monitor is a great for checking enum like columns.
Details
Cardinality is run via sql like: SELECT COUNT(DISTINCT <yourColumn>) FROM TABLE
Uniqueness
Description
Uniqueness tracks the percentage of unique elements in a given a column. A value of 1 would mean the column is 100% unique while a value of 0 would mean the column is 0% unique (e.g. all the same value). The monitor will alert if the value is higher or lower than expected. This monitor is good for validating primary key columns.
Details
Uniqueness is run via sql like: SELECT COUNT(DISTINCT <yourColumn>) / COUNT(1) FROM TABLE
Nullness
Description
Nullness tracks the percentage of null elements in a given a column. A value of 0 would mean the column is 100% non-null while a value of 1 would mean the column is all null. The monitor will alert if the value is higher or lower than expected.
Details
Nullness is run via sql like: SELECT SUM(CASE WHEN <yourColumn> IS NULL THEN 1 ELSE 0 END) / COUNT(1) FROM TABLE
Minimum
Description
Minimum tracks the lowest value of a numeric column. It will alert when the value is higher or lower than expected.
Details
Minimum is run via sql like: SELECT MIN(<yourColumn>) FROM TABLE
Maximum
Description
Maximum tracks the highest value of a numeric column. It will alert when the value is higher or lower than expected.
Details
Maximum is run via sql like: SELECT MAX(<yourColumn>) FROM TABLE
Mean
Description
Mean tracks the arithmetic mean of a numeric column. It will alert when the value is higher or lower than expected.
Details
Mean is run via sql like: SELECT AVG(<yourColumn>) FROM TABLE
Standard Deviation
Description
The standard deviation monitor tracks standard deviation numeric column. It will alert when the value is higher or lower than expected.
Details
Standard Deviation is run via sql like: SELECT STDDEV(<yourColumn>::double) FROM TABLE
Sum
Description
The sum monitor tracks the sum of a numeric column. It will alert when the value is higher or lower than expected.
Details
Sum is run via sql like: SELECT SUM(<yourColumn>) FROM TABLE
Percent Zero
Description
Percent Zero tracks the percentage of values equal to zero in a given numeric column. A value of 0 would mean the column is 100% zero while a value of 1 would mean the column has no zeroes. The monitor will alert if the value is higher or lower than expected.
Details
Percent Zero is run via sql like: SELECT SUM(CASE WHEN <yourColumn> = 0 THEN 1 ELSE 0 END) / COUNT(1) FROM TABLE
Percent Negative
Description
Percent negative tracks the percentage of values less than zero in a given numeric column. A value of 0 would mean the column is 100% negative while a value of 1 would mean the column is all positive or zero. The monitor will alert if the value is higher or lower than expected.
Details
Percent Negative is run via sql like: SELECT SUM(CASE WHEN <yourColumn> < 0 THEN 1 ELSE 0 END) / COUNT(1) FROM TABLE
Custom SQL
Description
The custom SQL monitor allows you to enter your own totally custom sql query, giving you the ability to create much more complex and specify monitors.
Details
If your query returns a scalar numeric value (e.g. 50), Metaplane tracks that value over time and alerts you if the value is higher or lower than expected. If your query returns a result set, Metaplane tracks the number of rows returned over time and alerts if the value is higher or lower than expected.
Updated 9 months ago