operations icon indicating copy to clipboard operation
operations copied to clipboard

Use StatusCake Heartbeat / Push Monitoring in backup scripts

Open Firefishy opened this issue 2 years ago • 2 comments

The OSM backup scripts should have better status monitoring.

An option is for the backup script on completion to call a StatusCake Push Monitoring endpoint with a "heartbeat". Each backup will have its own StatusCake Push Monitoring setup / endpoint.

The StatusCake Push Monitoring would alert if a heartbeat hasn't been called in a to-be-defined interval.

Firefishy avatar Sep 17 '23 22:09 Firefishy

We can probably get the filestat exporter to monitor the backup directory and use prometheus to alert if things are too old.

tomhughes avatar Sep 17 '23 22:09 tomhughes

Added initial test heartbeat code in https://github.com/openstreetmap/opentofu-statuscake/blob/main/heartbeat.tf Updates to backup script is pending, needs decision on what I do with the heartbeat ping URL, likely is a secret.

Firefishy avatar Feb 16 '24 19:02 Firefishy