sentry icon indicating copy to clipboard operation
sentry copied to clipboard

feat(crons): Backfill missed clock-tick dispatches

Open evanpurkhiser opened this issue 2 months ago • 1 comments

When we detect a clock tick it is possible that we may have skipped a tick when a monitor ingest partition is very slow and does not contain a message for a entire minute (and other partitions have already moved multiple minutes forward).

Previously we would log this and avoid producing a clock tick for these skipped minute(s) as we were using celery to dispatch the check_missing and check_timeout tasks. Since the celery tasks would be produced back-to-back it wasn't unlikely they would be processed out of order

Since the completion of GH-58410 we are now guaranteed that clock tick tasks are processed in order.

evanpurkhiser avatar May 06 '24 14:05 evanpurkhiser

Won't merge until we've enabled the consumer in US/DE

evanpurkhiser avatar May 06 '24 16:05 evanpurkhiser