Allow failures on Beta

[chriskrycho]

trafficstars

We need to make it so our builds are not failed by running against ember-cli beta (or typescript@next), even if we want to know that they're failing on beta.

[mike-north]

Currently the beta builds are not required in order to merge (and linux fixed, linux floating, windows, e-cli latest, ts latest are required).

Github only regards a commit as having a top-level "passing" status if all statuses are also passing. Unfortunately, with what I know about SCM/CI systems, tolerating some failures would basically involve burying those signals to where they're less likely to be noticed by us or other contributors.

What we can do is regard the "merge" button as our required CI indicator (in addition to the clear feedback above it). What do you think?

Required statuses pending

Required statuses passing

[chriskrycho]

I think there's an important distinction here around our use of CI tooling: between failure and a warning about upcoming changes that will affect us.

That includes, as I note in #688, things like our build status badge. As things stand, I would rather remove the badge entirely than have it reporting that our builds are currently failing simply because we're using CI to make sure we have advance warning about possible breakage for us.

Optimally, I'd prefer that we see it in PRs—precisely for the reason you noted on #688:

Personally, I want to see these failures. Particularly once the e-cli beta starts passing again, I definitely want it to be obvious if we're considering a merge that would cause it to start failing again. Do you have a different opinion?

But I don't want it to impact the status of the build. As a group of maintainers, if we know that a PR introduces a failure against beta, I don't think we would merge it. (I certainly wouldn't.)

[chriskrycho]

We've tweaked this to a spot we're all happy with – regardless of whether it exactly matches what anyone proposed here!