New Normal Annotations

We’re excited to share new annotation options for you to get an alerting monitor back into Normal state! These options replace the Mark as Normal option to help the model distinguish between two types of “normal”:

  • Use Until this happens again if there was a one-off incident that caused an alert, but the metric is at what you now consider a normal state. The monitor will alert you if similar incidents happen in the future.
  • Use Until there's a bigger anomaly if there was an alert for behavior that you consider normal and expect will occur again. The monitor will not alert you if similar incidents happen in the future.

As before, you never have to apply either of these - you can just wait for the metric to get back into the expected range. However, if you want to resolve active incidents or you don’t expect the metric to ever get back to the expected range, then these options will get you from red to green while helping the model improve.

If you're not sure which to choose, here is some more detailed information. But don't worry - if you change your mind, you can always go back and remove an annotation.

Until this happens again

How "new baseline" works in practice

  • The expected range will adjust temporarily to include the annotated value, but then revert to it's pre-anomaly behavior
  • How the expected range adjusts depends on the scenario:
    • For a big jump or drop, the range stays about the same size as before the anomaly, but moves up or down
    • For a row count flatline, the range expands to include the value
    • For a freshness monitor, the range stays large enough to include all observed values until the next refresh, then returns to the pre-anomaly size

Until there's a bigger anomaly

How "not a real anomaly" works in practice

  • This will expand the expected range to include the annotated value
  • The model will remember this forever so you don’t need to keep redoing it. Only choose this option if you are OK with never getting alerts for this type of behavior again
  • What the model remembers depends on the scenario:
    • For a big jump or drop, it will remember the size of the jump/drop
    • For a row count flatline, it will remember the length of the flatline until the value changes
    • For a freshness monitor, it will remember the amount of time until the next refresh
    • For a percentage monitor (nullness, uniqueness, % zero/negative), it will remember the observed value