Notify rules authors automatically when pipeline goes red

[keith]

Sometimes downstream / main branch jobs go red on their daily crons, it would be useful if an issue was automatically filed in this case so it could be triaged

// @Yannic

[philwo]

Buildkite has an e-mail notification feature that you could use to get such notifications for pipelines you're interested in. For rules_apple (just as an example) this can be setup via: https://buildkite.com/bazel/rules-apple-darwin/settings/email-preferences

I've been using this for the Bazel pipeline myself and it's pretty nice (except when your tests are flaky 😬 )

Could you let me know if that works for you?

[Yannic]

That would work for me, except that I don't have the permissions to access the settings, so I can't add my email for rules_proto or grpc-web.

Would also be nice if there was a way to get notifications when Bazel+downstream starts failing for a specific project (for HEAD versions or RC's of course, not for runs to check if a bazel PR breaks anyone)

[keith]

I do think that works for me but yea I guess I have enough access on the specific pipelines I care about

[keith]

Another case I realized was if the downstream build breaks here https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/2270 you couldn't do notifications per-pipeline like this even if you had access since there are too many different owners at once