buildkite
Slack integration build state filtering doesn’t filter unblocks
We want to alert our engineering Slack channel any time our master branch build fails (and when it is then fixed). When our master build is failing, teams are unable to release to production until the failure is resolved; this makes it worth alerting all our engineers so they can collaborate on a fix.
Buildkite’s Slack integration includes options to filter notifications on branch (e.g. master builds only) and build state (e.g. “Only send notifications for failed and fixed builds”). Unfortunately, this latter option does not work as expected when the build pipeline includes block steps.
Our master build pipeline ends with a couple of block steps to deploy the build to test environments. Normally master builds will run successfully, deploy to production, and then block. Engineers will often unblock one or both of these additional deploy steps. Buildkite posts notifications to Slack when these steps are unblocked, even though we have the “Only send notifications for failed and fixed builds” option checked.
This seems to be a pretty clear-cut bug to me. Unblocking a build step isn’t a failed or fixed build, so a notification should not be sent when build state filtering is enabled.
Because of this issue, we have had to disable our Slack integration (it was too noisy, so engineers were ignoring it). We are currently depending on engineers to proactively monitor the build status of their merges to master, which is not a sustainable solution.
Hi @sentience! :wave:
Yeah, sorry, this seems pretty frustrating. 😞 When we first introduced block steps the build was considered "finished" when blocked. We only added a "blocked" build state much later, and didn't want to break existing workflows so continued to call them "finished". But I reckon folks understand that "blocked" != "finished" these days.
We've been meaning to revisit our Slack integration for a while. I'm not sure when that will be, but I'll see if I can sneak in some better message filters quickly for you.
@sj26 We’re actually happy with the blocked == "finished successfully” thing. If we were to fail a build, and then fix it, the fixed build would block before the optional extra deploy steps, and we’d want to get notified of that as fix of the previous failure.
Ah, sorry, I meant that we mix up those states when they should probably be independently filterable for Slack notifications. I'll see if I can turn it into more checkboxes. Because who doesn't love more checkboxes.
