[Idea] Check notifications regardless of prior state (optional)

[frank-west-iii]

We've started using checks more often in our company since we are a small team, but the terminology threw me off a little when trying to accomplish a goal of getting a daily check on some info in our system.

We run a check every day to see if there has been data added to a table in the past day. Sometimes with this check it will follow the intended use of checks and reset itself from failing to passing and vice versa. However there are days where this check fails two days in a row. It's technically a new failure in the scope of the data, but not within the blazer check system.

We've solved this using custom rake tasks that wrap blazer and new columns in the blazer check table, but honestly I don't love touching areas that we don't own and we don't get a seamless UI experience in Blazer since we have to manually update the checks in the rails console to have them be typed so we can reset them before checking them.

Would be great to hear your insight on this. Maybe this could be a couple of new check types. i.e. missing_data_continual, bad_data_continual

[ankane]

Hey @frank-west-iii, you can use the blazer:send_failing_checks task to get a daily summary of failed checks (if that works for your use case).

[frank-west-iii]

Thanks. I have used that in the past. It might get us close enough for our use case if we could adjust the task to take the SCHEDULE environment variable like run_checks does.

Then I could still wrap the regular check task and follow it up with a failing check task run (for different schedule cadences).

It might result in duplicate emails in the cases where run_task and send_failing_checks sends the same notification, but that's not the worst thing in the world.

[frank-west-iii]

Hey @ankane, since it was a small change I wanted to put up what that would look like for me. No worries if this doesn't work for your vision. I have another potential solution for monkey patching this inside our rake task.

The implementation was ripped straight from run_checks

https://github.com/ankane/blazer/compare/master...frank-west-iii:failed-schedules