snowplow-rdb-loader
snowplow-rdb-loader copied to clipboard
RDB Loader: raise an alert in case of degraded Redshift
This is a follow-up for #608. In addition to just checking if loading is progressed via stages, we also should be able to leverage STV_LOAD_STATE docs claim it's available to all users. So, if we see that stages are progressing very slowly - we can check the table to make sure that progress is slow. And if it is - raise an alarm.
Additionally there should be a isDegraded flag in global state, turned into true when we discover a slow DB. Whenever it's true we should at least stop folders monitoring to not degrade the warehouse completely.