A better story for canary jobs (yellow CI)

[jonhoo]

This PR does two things.

First, it adds a new smoke test to the mix: cargo check -- -D warnings, which will error on any warning from rustc.

Second, it makes it easy to opt out of all canaries (basically anything that is allow_fail) by setting

parameters:
  canaries: false

This will currently disable:

• rustfmt on beta
• clippy on beta
• the new -D warnings test
• test on nightly

This parameter is intended for users that worry about long-running or highly parallel CI, yellow CI ("partially successful").

[djc]

FWIW, I like the idea of (optionally) erroring stable CI for warnings. I think denying warnings at compile time is too annoying for my development experience, but I want code to be warning-free before it hits master.

[jonhoo]

@djc there are two tricky parts about this:

• Commits that did not emit warnings in the past may start emitting warnings in future versions of stable, which would cause a CI failure in, say, an unrelated PR.
• Semver-compatible changes in dependencies (macro changes and deprecations in particular) can cause a previously-fine commit to suddenly stop passing if you don't check in Cargo.lock.

These taken together mean that failing CI on warnings, even on stable is risky business. The open discussion above is basically about what policy to take on this. It might be possible to have a "error on warnings with a pinned Rust version + allow particular warnings related to dependencies", but it's finicky and somewhat brittle.

[codecov-io]

Codecov Report

Merging #33 into master will not change coverage. The diff coverage is n/a.

@@          Coverage Diff          @@
##           master    #33   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           1      1           
  Lines           8      8           
=====================================
  Hits            8      8

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 9e8e64f...4e6377a. Read the comment docs.