Faster release cadence for bug fixes to stable

[lopopolo]

Description

In my projects I upgrade to the latest Rust toolchain quickly after release. It is fairly common to have to work around bugs in clippy as part of this process. From browsing the issue tracker, false positives and bad suggestions tend to get reported in batched when a new stable compiler hits.

Here are some such issues that I have filed or been interested in:

• https://github.com/rust-lang/rust-clippy/pull/5998#issuecomment-731855891
• https://github.com/rust-lang/rust-clippy/issues/6137
• https://github.com/rust-lang/rust-clippy/issues/6739
• https://github.com/rust-lang/rust-clippy/issues/6970
• https://github.com/rust-lang/rust-clippy/issues/8120
• https://github.com/rust-lang/rust-clippy/issues/8281
• https://github.com/rust-lang/rust-clippy/issues/8300
• https://github.com/rust-lang/rust-clippy/issues/8470
• https://github.com/rust-lang/rust-clippy/issues/9321

I love that clippy has a wide selection of lints, but the reality is some of them are buggy when they get released. Even if I report a bug the day a stable compiler comes out, it takes at least 12 weeks for fixes to roll out (for fixes to go through a full nightly to beta to stable cycle).

Is there a way to roll out fixes for false positives and bugs in stable clippy more quickly?

I've previously expressed concerns about buggy lints making it to stable in https://github.com/rust-lang/rust-clippy/pull/5998#issuecomment-731855891 where I proposed running new lints through something like a crater run. After thinking about it, I think what I'd actually like to see is bug fixes be released in a more rapid manner than they are today.

I've also previously asked if clippy bugs getting fixes would warrant a point release to the stable Rust toolchain. I'm not sure what the team's thinking on this is.

• https://github.com/rust-lang/rust-clippy/issues/8470#issuecomment-1050555911

Version

rustc 1.63.0 (4b91a6ea7 2022-08-08)
binary: rustc
commit-hash: 4b91a6ea7258a947e59c6522cd5898e7c0a6a88f
commit-date: 2022-08-08
host: x86_64-apple-darwin
release: 1.63.0
LLVM version: 14.0.5

Additional Labels

@rustbot label +C-question

[xFrednet]

This is a problem we're trying to tackle and is part of Clippy's roadmap. See https://github.com/rust-lang/rust-clippy/issues/6623

Currently, there are some initiatives/ideas floating around that we want to implement.

1. We hope to add a new testing period to new lints where they are only enabled on the nightly tool chain for one or two releases. (This will hopefully be my next project after #[expect] was stabilized)
2. There was some suggestion to run lintcheck, which is basically a mini crater with 15 crates or so, on every PR before the merge. This can be expanded to running a larger dataset on releases. There are actually projects which already run Clippy on 1000 crates from crates.io. Adding this to our release planning could be a good idea.

cc: @Alexendoo FIY I believe you wanted to better integrate lintcheck into the workflow :upside_down_face:

[lopopolo]

Thanks for the context @xFrednet. I'm excited that there is work planned to make a dent in this!

In the absence of having nightly only lints, might we consider a policy that all new lints are placed in nursery? It's hard for me to tell what the nursery lint group is for, but this might be an easy way to stop the bleeding in the short term.

[xFrednet]

This was something we thought about before. The problem is, that we also add restriction lints and others that shouldn't be warn-by-default. Therefore, users couldn't say #![warn(nursery)] which would mean that we still don't get any feedback until we enable them. It just pushes the problem down the road.

(#![warn(nursery)] might be bad style anyways, but some few crates use it as most lints mostly work and would not be in the restriction category.)

[est31]

@lopopolo you could try using nightly clippy at least for local development. Then a) the bug fixes will arrive to you in particular more quickly, and b) stable users might not even notice that a bug has existed at all.

[lopopolo]

I'm not sure how that would help since it looks like in some case bugs are reported on nightly and don't get fixed by the time the lint makes it to stable:

• https://github.com/rust-lang/rust-clippy/issues/8961

@est31 In theory I could run a weekly job against nightly clippy (across 15 repositories and 30+ crates will not be fun though) but my preference would be for clippy to do its own testing.

[xFrednet]

Running nightly builds on a schedule would help us a lot. If we noticed that new lint's produce several FPs we try fixing or move them to an allow by default category (often nursery). For moves we also often Backport the change to the beta branch if required.

[lopopolo]

Another class of issue I recently ran into is that lints may be introduced as part of one lint group (e.g. pedantic) and downgraded to another level (e.g. restriction). Sometimes this happens before the lint reaches stable but after the beta cutoff. In cases like this, it tends to be the case that the downgrade to another lint group is not cherry picked to the beta branch which leads to a churn-inducing lint making it to stable.

See:

• https://github.com/rust-lang/rust-clippy/pull/10442