hyperlane-monorepo icon indicating copy to clipboard operation
hyperlane-monorepo copied to clipboard

Alert query for inconsistent latest checkpoints doesn't seem accurate

Open tkporter opened this issue 1 year ago • 0 comments

Problem

See https://abacusworks.grafana.net/alerting/e26839dc-2f4c-4ff3-9e31-734dbf9cf061/edit?returnTo=%2Falerting%2Flist

  • The first part of the query takes a max of the same labels twice
  • The second part of the query takes a min of the same labels of a max - doing nothing
  • Seemingly this will never fire. We should've expected to see this with the config issues around the SG-1 validator's announcement (see https://discord.com/channels/935678348330434570/1240968168172814417)

Solution

Write the query such that we alert for cases where there are repeated strong inconsistencies in a validator set's latest index

tkporter avatar May 17 '24 10:05 tkporter