once_cell
once_cell copied to clipboard
matklad
Minimum Rust Version
The latest version of once_cell bumps up the minimum requirement up to 1.56 which is a huge jump. Is it conceivable to do a major version bump for this so that people who want to depend on it for libraries that wan to support older versions can stay on the old version?
The short answer is no: per current policy, MSRV bump is not considered a semver breaking change. I want to avoid every reverse dependency having to change their Cargo.tomls (and their own major version, if they happen to expose once_cell via a public API).
After re-reading https://github.com/rust-lang/libs-team/issues/72 couple of times, the policy I decided for once_cell is:
We support workflows which target up-to-date, supported versions of Rust compiler, but we also give somewhat generous grace period, as keeping perfectly up-to-date is hard.
We explicitly do not support workflows which depend on using the old compiler. Making sure that the latest once_cell can be compiled with rustc packaged with debian stable is a non-goal.
If users of once_cell find themselves with an outdated compiler, the following actions are suggested:
- Find a way to upgrade compiler
- If using old compiler is required, stick to old version of once_cell as well
- If a combination of old compiler and new once_cell is required, it's on the consumer to maintain a fork with backports.
The following considerations were the most salient for me:
- Rust's own policy is essentially "live at head" -- only the latest rustc is officially supported
- The number of people who are stuck with the old compilers is relatively small
An orthogonal policy is that MSRV bump is not considered semver breaking. There are two justifications for that:
First, this creates ripple effect over ecosystem, where each reverse-dependency has upload a new version with a different Cargo.toml. If the crate also happens to be part of the public API or pushes revdep's own MSRV, the reverse dependency is required to bump its major as well. Practically, not every rev-dep will notice new once-cell, so an extra burden falls on folks maintaining applications with Cargo.locks, who now need to chase their upstreams to avoid duplicate entries in Cargo.lock.
Second, bumping sevmer on MSRV makes the ecosystem less compatible with older compilers. Consider once_cell 1.0.0 with 1.42 as MSRV and kittens 1.0.0 which depends on once_cell = 1 and has it's own MSRV of 1.48. Now, once_cell bumps its MSRV to 1.56. The two scenarios are:
A: once_cell releases 2.0, kittens updates its dep to 2.0 and releases kittens 2.0 with MSRV of 1.56. Then, kittens releases 2.1 with a new API. A binary project rip_toys wants to use this new kittens API, so it upgrades to 2.1. Now, it's impossible to use the latest version of rip_toys on debian stable, because of MSRV.
B: once_cell releases 1.1. kitten does nothing: it still supports 1.48 when Cargo.lock contains once_cell = 1.0.0. When kittens adds a new API, it publishes 1.1.0. rip_toys uses the new API, but it keeps once_cell=1.0.0 in its Cargo.lock. As a result, newest version of rip_toys stays compatible with debian stable.
To rephrase this more mathematically, MSRV means that there exists a combination of dependencies which can be build with that compiler, not that any combination of dependencies can be build with it.
The exists formulation gives more freedom for actual users with Cargo.locks to pick their deps.
One practical problem with the above is that, if you naively test MSRV on CI, your CI might now be broken by upstream releasing a new version to crates.io. To solve this problem, the following rule is used:
MSRV CI: if you are not using the latest stable compiler, you must use Cargo.lock.
The suggested way to do this is to commit Cargo.lock.msrv file to the repo, and cp Cargo.lock.msrv Cargo.lock when running CI for MSRV. Here's how once_cell itself dose that:
https://github.com/matklad/once_cell/blob/97edd07e0ac05fb8e4b994e9a53ba187d8fa17fc/xtask/src/main.rs#L43-L48
Finally, there are three specific things anyone feeling strongly can do to improve the situation:
- work on stabilizing https://github.com/rust-lang/rust/issues/74465
- teach cargo to use
rust-versionduring version selection - write a gum & duct tape script which calls
cargo update --precise && cargo checkin a loop for creating Cargo.lock for old compilers without direct support for rust-version in Cargo.toml.
Given that this is where the ecosystem is going, I will likely have to stop maintaining my own libraries for old rust versions.
teach cargo to use rust-version during version selection
I can't even imagine how this works in the resolver given the complexities of this. The resolver would have to back out a huge chunk of the dependency graph if it first encounters an incompatible version of a library.
I think the practical implication of this is that the community starts to restrict to test only against latest Rust again or maybe some quite recent version.
One question I have here: today, once_cell's MSRV is described as "conservative" in readme and 11 months for me does sound conservative, though pretty close to the boundary. I'd like to get a rough "temperature reading" to use the words in a useful way:
Could 11 months old MSRV be called conservative?
- :rocket: : if yes
- :eyes: if no
One question I have here: today, once_cell's MSRV is described as "conservative" in readme and 11 months for me does sound conservative, though pretty close to the boundary. I'd like to get a rough "temperature reading" to use the words in a useful way:
Could 11 months old MSRV be called conservative?
I think my answer would depend on whether 11 months is policy. Your long comment above to me suggests a more aggressive policy (on when we can expect MSRV updates) but as far as I can tell it doesn't actually specify when/how future updates would occur. I would agree your current choice of MSRV is (somewhat) conservative.
It's a shame that MSRV update is not part of the semver breaking changes as it completely broke my builds for sysinfo (as you can see in https://github.com/GuillaumeGomez/sysinfo/pull/844). My two cents here would be that making a major release for MSRV change is better, especially for ecosystems where you stick to a precise rustc version.
@GuillaumeGomez as per MSRV CI rule articulated above, ecosystems using precise rustc version could also use precise Cargo.lock to avoid any breakages.
But that would prevent to automatically get any bugfix too. Tricky situation. :laughing:
Applications generally use a lock-file anyway, so they already don't get automatic bugfixes.
For libraries, you want to run CI on stable without lockfile (to catch new upstream regressions) and on MSRV with lockfile (to catch MSRV/min-version regressions in your own code).
@GuillaumeGomez I'd recommend https://github.com/dtolnay/rust-toolchain#toolchain-expressions instead of lockfile. This should be compatible with once_cell as implied by "we give somewhat generous grace period, as keeping perfectly up-to-date is hard" as long as the number of months/releases you pick for the CI build matches what once_cell would consider somewhat generous.
No, I'll just enforce in my Cargo.toml to use the previous version so I can keep the current MSRV in sysinfo.
Applications generally use a lock-file anyway, so they already don't get automatic bugfixes.
Except when installing through cargo install, which will default to resolving dependencies from scratch.
No, I'll just enforce in my
Cargo.tomlto use the previous version so I can keep the current MSRV in sysinfo.
This is the worst approach that's been mentioned so far. This blocks anybody from using new sysinfo features together with new once_cell features, even if their compiler supports it.
I don't see how blocking the once_cell version used in sysinfo is impacting anyone using sysinfo. When I'll make a new major release, I'll update the once_cell version alongside the MSRV and that's it. It's really problematic that a minor update can break your compilation.
Also I just realized that it was used in other dependencies of sysinfo, so basically I can't do anything about it in here... This is really bad... Why not bumping the medium version or something? Even with all your explanations, I don't understand how semver can allow this breakage inside minor versions. Doesn't make any sense...
So basically, I'm now forced to update MSRV version of sysinfo (and make a new major release) because a dependency decided to make a minor release with a breaking change.
To be clear: I'm not opposed to a MSRV change, I'm opposed to a silent MSRV change.
I don't understand how semver can allow this breakage inside minor versions. Doesn't make any sense...
I can try to answer this. Semver very explicitly only applies to the documented public API. You first explain how your thing is intended to be used, and then everybody using it in that way can use the semver number to tell whether a particular update is potentially breaking.
A great number of observable things are not covered by semver because they are not documented as part of the intended way to use a crate. For example code that reaches into doc(hidden) private macro internals doesn't get to complain when those things change, because they are not public API. Similarly code that transmutes types with private members. Or code that assumes undocumented implementation details of layout, such as the precise size of a type in bytes.
In once_cell's case, the "public API" is that you build it with a recent enough compiler and stick to everything shown in the rendered rustdoc, and then once_cell's semver number will tell you which releases potentially break that usage.
Thanks for the explanation. We all agree here that @matklad respected semver. What I'm complaining about here is that even though semver doesn't go over this, if the same code with no modification stop compiling because of a dependency, then it is a breaking change.
Well, I think the situation won't change and @matklad won't make a medium/major release over this so I'll just wait for a few days and then make a new major release for sysinfo like I said previously. Please just note that this is a very frustrating situation.
So, whether MSRV bump should be considered semver breaking is surprisingly contentious topic. I think this happens because ecosystem effects of bumping semver in this case are not obvious. However, once ecosystem-wide implications are understood, I think it becomes rather clear that bumping semver with MSRV is untenable.
I've tried to reason through the effects in the above comment, but that's still is a bit too long-winded. So let me try to skip the deductions, and just lay out the conclusions:
If MSRV bump is a breaking change, then:
- major bumps happen significantly more frequently for "leaf" libraries
- major bumps happen constantly for intermediate libraries, their authors are on a release treadmill due to dependencies even if they make zero changes to the library's own code
- Cargo.locks on average contain a lot of duplicate crates
- Rust applications can be built only with fairly recent versions of Rust
If MSRV bump is not a breaking change, then:
- intermediate library authors have to have marginally more complex CI setup for MSRV testing
- intermediate library authors don't care when their deps bump MSRV
- Rust applications stay compatible with wider range of Rust versions, though actually getting an application to compile with old Rust requires at the moment semi-manual futzing with Cargo.lock
I don't see how blocking the once_cell version used in sysinfo is impacting anyone using sysinfo.
This is an example of subtle ecosystem implication.
Let's say sysinfo 1.2.3 has once_cell = ">= 1.2.0, < 1.15.0" in Cargo.toml. Let's say foo has sysinfo = "1.2.3" in it's Cargo.toml, and bar has once_cell = "1.15.0". Both foo and bar works fine in isolation. However, if an app writen by someone else depends both on foo and bar, it just fails to build now, on any compiler, because there are no Cargo.lock which satisfies dependencies.
If all dependency requirements are unconstrained above (eg, of the form = "x.y.z"), Cargo can always satisfy all deps by just greedily picking the latest minor of every relevant major (aka, rsc's argument that minimal versions dodge NP-hard also works for maximal versions. What matters is not the direction of lattices, but absence of non-open constrains)
If we add upper-bound constraints (= "=x.y.z", = "<x.y.z"), then some dependency graphs become genuinely unsatisfiable.
@matklad I think your above points are well reasoned. I'm curious about one thing: Do you think anything changes for a library that's still in a 0.y.z version? My own strict reading of semver rules concludes that that nothing in your analysis changes, but a less strict reading of the statement "Anything MAY change at any time" suggests that maybe a library that is not yet at version 1 might be justified bumping the minor version when changing MSRV. (But even if this is true, I don't think this observation really helps our situation here)
Maybe I'm just alone thinking that. Well, if it's supposed to be this way, there's no point in opposing it then. Thanks to everyone for your answers! I'll just bump the sysinfo MSRV and make a minor release to prevent more breakage.
Do you think anything changes for a library that's still in a 0.y.z version?
Mechanically (ie, what actually happens in the guts of Cargo), 0.y.z works as y.z.0, there's nothing special there.
Culturally, I don't think it is true in Rust ecosystem that 0.y.z = anything goes. I'd say we generally treat 0.x versions as x.0 versions, but it always is a good idea to consult crate's doc to avoid second-guessing authors intentions.
Culturally, I don't think it is true in Rust ecosystem that 0.y.z = anything goes. I'd say we generally treat 0.x versions as x.0 versions, but it always is a good idea to consult crate's doc to avoid second-guessing authors intentions.
This is correct. While formal semver specifies that 0.x.y is anything goes, cargo explicitly uses a different behavior, where 0.x.y is compatible with 0.x.(y+1):
This compatibility convention is different from SemVer in the way it treats versions before 1.0.0. While SemVer says there is no compatibility before 1.0.0, Cargo considers 0.x.y to be compatible with 0.x.z, where y ≥ z and x > 0.
Sigh... It's the second time this week when a fundamental dependency broke MSRV contract of downstream projects. First it was time and now this.
@matklad
So let me try to skip the deductions, and just lay out the conclusions
You forgot about the third way: MSRV-dependent dependency version resolution. This way MSRV bump will not be a breaking change and will not break downstream builds reliant on older toolchains. Unfortunately, it does not look like anyone works on it...
@newpavlov Neither time nor once_cell violated the MSRV contract. I have already explained this to you for time. Please keep the discussion in time-rs/time#484, where it has been taking place (there's no need to bring an unrelated crate into this repo). There is at least one open question from me from a couple days ago. once_cell says it will be "conservative". Are you asserting that waiting almost an entire year after a feature lands on stable is not conservative? If so, what would you call conservative?
Further, @matklad has already stated that MSRV-dependent resolution is an option:
specific things anyone feeling strongly can do to improve the situation: … teach cargo to use rust-version during version selection
The fact that no one is actively working on this is, frankly, not an issue that concerns myself and @matklad (I presume). Anyone is able to do this. Nearly everyone working on Rust is a volunteer. If you would like to see MSRV-dependent resolution, ask on Zulip where a plausible starting place is.
@newpavlov note that it never was once_cell policy’s to support Debian stable; 10 versions back was as far as we went, and that’s not enough to support Debian.
If the MSRV contract of a downstream is that it supports Debian stable with any combination of dependencies, the project became buggy the moment it added once_cell to its list of transitive deps. Please avoid using once_cell in such projects.
Note that this doesn’t mean that it’s impossible to use once_cell with anything targeting Debian stable. It just means that projects doing so should use sufficiently lax lower bounds on once_cell to get by with sufficiently old version, provide a Cargo.lock with this version, and apply due diligence checking that that version wasn’t flagged with any security vulnerabilities.
@jhpratt
I did not say time and once_cell broke their MSRV contract (sure, since they do not have one). I am talking about downstream projects which strive to keep MSRV contract and crates like time and once_cell undermine such efforts. Some projects even chose to simply remove once_cell from their dependency tree because of that, which is obviously far from ideal.
If so, what would you call conservative?
The approach which I consider conservative and which I personally follow: consider MSRV bump a breaking change until MSRV-dependent version resolution lands and crate's MSRV becomes higher than version at which the feature has landed.
@matklad
If the MSRV contract of a downstream is that it supports Debian stable with any combination of dependencies, the project became buggy the moment it added once_cell to its list of transitive deps. Please avoid using once_cell in such projects.
I agree and as we can see some projects have chosen to act on your last sentence. I would hope that foundational crates would adapt a more conservative stance, but alas it looks like such hope is a vain one...
consider MSRV bump a breaking change
Even setting aside that bumping MSRV is largely accepted as a minor version bump, not major, there are enormous consequences to a change like this. matklad has already listed these in this thread.
I would hope that foundational crates would adapt a more conservative stance
libc is looking at a much less generous policy than either crate you have mentioned. Repsectfully, you're in for much larger issues if you think time and once_cell have caused MSRV-related issues.
One question I have here: today, once_cell's MSRV is described as "conservative" in readme and 11 months for me does sound conservative, though pretty close to the boundary. I'd like to get a rough "temperature reading" to use the words in a useful way:
Could 11 months old MSRV be called conservative?
Well, it could be called that, but I would not really consider it to actually be conservative.
Why is that? The term "conservative" in relation to software versions is usually closely related to the notion of Long Term Support (LTS) of software / OSes / whatever component we are talking about. Because that is what "conservative" in versioning boils down to for me: being able to use new versions of a crate without having to worry about whether an update to the newest toolchain version is required with every new version of a Rust crate. Since Rust has no notion of LTS, let's take a look at the release cycle of Ubuntu, a popular Linux distribution which does have LTS releases, and which in turn may give us an idea what time span may be considered conservative. In essence, Ubuntu provides a new LTS release every two years with support for five years (plus additional five years, if one is willing to pay for that), and between those LTS releases it provides interim releases every six months which are supported for nine months.
Staying with that example, I would consider users of Ubuntu LTS releases conservative, and users of the interim releases not. If we look at the support time frame of five years for LTS releases, then I understand that supporting up to five year old software is not really feasible in most cases. But supporting software versions until the next LTS release is available, which in that case is two years, probably is feasible.
Using those time frames as a reference, I myself would consider supporting up to two years old software to be conservative. There certainly is room for discussion whether two years is too long or too short a time frame. Some but I would certainly not consider any time frame below one year as conservative.
To translate this back to Rust versions:
- The current stable release is Rust 1.64.0, released on 2022-09-22.
- The oldest release that still is within a one year limit from today is Rust 1.56.0, released on 2021-10-21, and this is also the first version to support the 2021 edition of the Rust language.
- The oldest release that still is within the two year limit is Rust 1.47.0, released on 2020-08-10.
So to summarize it: As of today (2022-09-26), supporting Rust versions back to Rust 1.47 is what I would consider really conservative. Supporting Rust back to Rust 1.56 is where I could start arguing whether this may be considered conservative. And any cutoff at a newer Rust version basically conveys to me that a project clearly does not care about a conservative MSRV policy. Just my two cents.
This is breaking one of our repo as well, does anyone know what's the best way of fixing this? In our case it's a transitive dependency so we can't set the version to =1.13.1 or something. The following doesn't seem to work either:
[patch.crates-io]
once_cell = "=1.13.1"
(BTW our problem comes from the bump in Rust edition)
