rust-lang
RFC: Allow packages to specify a set of supported targets
Summary
The addition of supported-targets to Cargo.toml. This field is an array of target-triple/cfg specifications that restricts the set of targets which a package supports. Packages must meet the supported-targets of their dependencies, and they can only be built for targets that satisfy their supported-targets.
I think that there should be a little more explanation about how docs generation works with this. Specifically: Can I build docs for a target that's unsupported, such as if my host machine isn't supported, can i cargo doc to still just see the documentation?
This would be a godsend for crates that link against third party libraries, thus limiting supported targets.
I think that there should be a little more explanation about how docs generation works with this. Specifically: Can I build docs for a target that's unsupported, such as if my host machine isn't supported, can i
cargo docto still just see the documentation?
Indeed, and this is especially important since docs.rs needs to be able to generate the docs for all crates. I added it here.
I skimmed so apologies if I didn't see it but suppose some crate decides to use this, specifies their list of targets, and then the maintainer walks away. Then down the road a year later a new target is added to Rust, and that target would be fine with the crate, but the maintainers aren't around to publish a new version.
I can't come up with a good solution to that, and to me if there isn't a good solution to that then this should be considered a non-starter. In a way it'd almost be like the left-pad fiasco, except that it happens automatically behind the scenes. As I read this, such packages are "deleted" from the perspective of targets they don't support. If one of those makes it into everything ALA left-pad style propagation then Rust can't get new targets.
As examples of recent targets loongson and various forms of webasm come to mind. It's not exactly a rare operation to add a new target.
The only thing I can think of is to warn/advise about this case by default. I don't see how any long term good comes from error by default, even if it short term solves some annoying problems.
Crates assuming that they're running on one of several targets could even end up making unsafe code decisions based on that fact. Forcing the code to "just build anyway" would naturally lead to problems.
And crates can already force themselves to only build only on a specific target, this would not be a new ability, but instead it's only a way to better organize that information.
The author already mentions in the Prior art that the following Rust can be written:
#[cfg(target_arch = "lol"))]
compile_error!("experience bij)";
Please do not make comments on the RFC which do not engage with the RFC's content.
Let's go ahead and start the process of asynchronously checking for consensus.
@rfcbot merge
Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:
- [ ] @0xPoe
- [ ] @Eh2406
- [ ] @Muscraft
- [ ] @arlosi
- [ ] @ehuss
- [ ] @epage
- [x] @joshtriplett
- [ ] @weihanglo
Concerns:
- field-name (https://github.com/rust-lang/rfcs/pull/3759#issuecomment-2580866191)
- should-crates-have-to-set-supported-targets-to-match-their-dependencies (https://github.com/rust-lang/rfcs/pull/3759#issuecomment-2580240283)
- should-this-be-in-manifest-or-updatable-metadata (https://github.com/rust-lang/rfcs/pull/3759#issuecomment-2581681334)
Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!
See this document for info about what commands tagged team members can give me.
@rfcbot concern should-crates-have-to-set-supported-targets-to-match-their-dependencies
@rfcbot concern field-name
See https://github.com/rust-lang/rfcs/pull/3759/files#r1909195446
I think the chances that this whitelist is far too restrictive is incredibly high. I've experienced a bunch of cases where the author of a crate says "I only test for x86 and ARM so these are the only officially supported targets" when the crate works perfectly fine on many other architectures (the code doesn't use anything architecture specific, the author just doesn't care about setting up CI for e.g. PowerPC). I would want to see some fallback mechanism that the author of a crate can specify such that in the example x86 and ARM can be the "officially" supported targets but other targets can still build just fine, but possibly emit a warning that they are not officially supported by the author.
Alternatively the documentation should strongly suggest not using the supported-targets in these cases.
Update: I just noticed another example of this. I'm the author of a hotkey listening crate. It supports specific operating systems. However, there's a dummy implementation for all other targets. It compiles, but no hotkeys are ever triggered. Here it would also be great to advertise in the Cargo.toml that Windows, Linux and co. are the main supported targets, but that it's perfectly fine (with possibly a warning to the user) to just use the crate across all targets.
Maybe if we adopt this it should include a way to describe whether the use of the crate on an unsupported target is one of
- actively dangerous (implementations depend on target-specific details that must be ported)
- probably bad? but not automatically unsound (implementation depends on target-specific details that work if another target is similar enough)
- maybe it works, maybe it doesn't? glwt (the common "I've only tested it on..." situation)
@workingjubilee I have read most of the comments (new ones are coming in too rapidly) and cannot find a thread to reply to where this point is appropriate. I'll be honest that I'm very surprised how much attention this RFC is getting.
I feel the need to push back on you (I believe it was you but GH won't let me see this?...) dismissing my previous comment as off topic. I am aware that you can technically do this today. I have now read the RFC in detail. The ability to do it today if you really want is not by itself a justification for encouraging doing so. Many targets are supersets of previous targets. I do not want crates.io to slowly build up this cruft. You can model this new metadata as effectively a gradual deletion of the package, and if you do so it's concerning. This kind of thing cascades to all downstream packages, without a good mechanism for direct control, and as far as I can tell introduces the first "blessed" mechanism into the language which in effect requires maintainers to remain around. Add this, encourage people to use it, decide that such things are good code, and the ecosystem is one popular package away from Rust not being able to easily get new targets anymore. Looking at the output of rustc --print target-list even today, we can identify groups like this:
arm-linux-androideabi
arm-unknown-linux-gnueabi
arm-unknown-linux-gnueabihf
arm-unknown-linux-musleabi
arm-unknown-linux-musleabihf
Obviously the correct thing in such a case is to use cfg predicates to account for the entire groups..but will everyone do that in practice? I doubt it.
What about Nintendo Switch? Who's going to even be able to test that one given that you need an SDK from Nintendo? There's targets like that in here where the original authors of packages literally cannot get access but their code might very well have been fine.
That said this is way too noisy and you're the mod and I've made my viewpoint clear, so I'm unsubscribing for now. But I ask that if you still think I'm off topic, please give more of an explanation why this isn't a legitimate, important concern. I personally consider most of the rest of the RFC to have low relevance until this question is answered, and am quite surprised that it wasn't answered before posting it. I may check back in later once this has had a chance to settle, but the chances of me properly keeping up at this point are very low so it doesn't seem worth me trying to do so now given that the nature of this concern boils down to a yes-or-no question more than a long drawn-out discussion.
- getrandom: https://github.com/rust-random/getrandom/blob/9fb4a9a2481018e4ab58d597ecd167a609033149/src/backends.rs#L156-L160 - This one is particularly notable because their implementation for many targets is actually identical due to using either
getrandomorgetentropy.
@rfcbot concern should-this-be-in-manifest-or-updatable-metadata
We might want to consider the possibility of using the proposed mechanism for updatable crate metadata. That would allow updating this list without uploading a new version of the crate.
On the other hand, that would be less convenient and less self-contained.
EDIT: See https://blog.rust-lang.org/inside-rust/2024/03/26/this-development-cycle-in-cargo-1.78.html#why-is-this-yanked for more about updatable crate metadata.
We might want to consider the possibility of using the proposed mechanism for updatable crate metadata. That would allow updating this list without uploading a new version of the crate.
If that metadata is update-able, what happens when some popular crate's maintainer suddenly decides to go rogue and they replace the set of supported targets with cfg(false) or equivalent? That would effectively be another leftpad incident.
There are several comments here that suggest it might be a problem if a crate sets this metadata and someone wants to run the crate on an unsupported target. I think it's worth having some clear documentation about recommended usage of this mechanism.
This should not, in general, be used for "I haven't tested this on other targets and I don't know if it works". This should be used for "I have good reason to believe it doesn't work as expected".
We might want to consider the possibility of using the proposed mechanism for updatable crate metadata. That would allow updating this list without uploading a new version of the crate.
If that metadata is update-able, what happens when some popular crate's maintainer suddenly decides to go rogue and they replace the set of supported targets with
cfg(false)or equivalent? That would effectively be another leftpad incident.
"What happens if a crate maintainer goes rogue" is not a question in any way specific to this mechanism. A crate maintainer could also replace the entire code of a crate with a compilation error, or worse.
If that metadata is update-able, what happens when some popular crate's maintainer suddenly decides to go rogue and they replace the set of supported targets with
cfg(false)or equivalent? That would effectively be another leftpad incident."What happens if a crate maintainer goes rogue" is not a question in any way specific to this mechanism. A crate maintainer could also replace the entire code of a crate with a compilation error, or worse.
except that currently they can't -- they can upload a new rogue version, but they can't do anything to existing versions except yank them (which afaik doesn't break users who have it already in their lockfile). update-able metadata allows them to break existing versions -- though if that metadata was also stored in the lockfile and wasn't updated unless running cargo update or equivalent, that could fix most of the problem.
@programmerjake I would expect it to be tied to cargo update, yes. But either way I don't think it's an issue we need to worry about.
I'm putting this as a top-level comment because there are some discussion threads above that might be talking about it but I'm not sure and I don't want to derail them if they aren't.
As a crate author one of the most important reasons why I want this feature is because it might enable pruning of my crate's transitive dependencies. Concrete example: I'm working on a crate that is inherently Linux-specific (it provides an interface to a Linux-specific kernel API). This crate uses rustix to make system calls. rustix itself is not Linux-specific, and in fact, despite the name, it can be used on non-Unix systems like Windows and WASI. For a while last year, rustix had transitive dependencies on two different versions of the windows-sys crate (directly and via libc) which caused clippy's "multiple-crate-versions" lint to flag my crate, even though my crate is never going to work on Windows. (See https://github.com/bytecodealliance/rustix/issues/1233 for more detail. This specific problem has been fixed but you can see how things like it are liable to recur in the future.)
So, because of that experience, I really want supported-targets = [...] in my crate to mean that Cargo prunes the dependency tree very early -- during the initial computation of Cargo.lock, probably -- such that cargo tree --target all and cargo tree both report only the transitive dependency set that is relevant to the declared supported targets, and clippy doesn't even consider what rustix depends on when it's being used on other targets.
At the same time, I also think it's very important for supported-targets to be override-able. It should be possible for someone to force a build of my crate on Windows, even though this isn't going to accomplish anything useful, with no more ceremony than e.g. cargo build -- -A cargo::unsupported-target. And it should be possible for some other crate to declare a dependency on my crate even if there is no overlap whatsoever between that crate's supported targets and mine. I may not think that's a useful thing to do, but I shouldn't get the last word here! Maybe it's 30 years from now and I'm dead and never going to update my crate again, but meanwhile "Windows" and "Linux" have converged to such an extent that you can usefully use it on Windows! Of course the long-term solution in that hypothetical is for someone to take over maintenance of my crate and update the supported targets list, but in the short term, it's the depending crate that should get the last word.
I bring these two things up at the same time because I see potential for a conflict between them. Cargo should prune the dependency tree early -- but it needs to do the pruning in a way that honors any override that is happening, and I'm not sure exactly what that turns out looking like.
In order to reconcile the desire for pruning with the desire for overrides:
We could make overrides be an entry in Cargo.toml about a package whose supported-targets should be ignored. Then, Cargo can take that into account when making the lockfile.
(Ideally, it would have to be in the top-level crate that cargo is currently being run on.)
I realize that some would like an override that's as easy as a command-line option, but I do think that would be in tension with having it affect lockfile generation.
(I also realize we have some command-line options that can affect lockfile generation, but that seems like a pattern we should avoid propagating.)
We might want to consider the possibility of using the proposed mechanism for updatable crate metadata. That would allow updating this list without uploading a new version of the crate.
For those that never heard about this mechanism, do you have a link with further details on this "proposed mechanism"?
Mutable metadata is talked about in https://blog.rust-lang.org/inside-rust/2024/03/26/this-development-cycle-in-cargo-1.78.html#why-is-this-yanked
In my company we've been developing embedded code for about 5 years for arm64, arm, amd64 and RISC V running macOS, Linux, Windows, WASM and bare metal so I feel I'm the target audience.
The two things that we needed were platform-specific deps (solved with [target.'cfg(target_os = x)'.dependencies]) and platform-specific features-specifically, making some features default-available on some platforms (unsolved as of yet, you can't do [target.'cfg(target_os = x)'.features]).
We have never had a crate not compile on a system and not understand why just by looking at the crate's name. We have never had a crate compile but work incorrectly on a system. I don't understand the RFC's section on motivation well enough to understand if this will help me.
I also don't understand what supported-platforms would signal. I see two options.
- This code compiles on the following platforms.
- This code has been verified to work on the following platforms.
For 1 if we're using it basically to cache the results of compilation the obvious issue is running out of sync. If we're not compiling because of a dependency and they fix the dependency, now our package won't even attempt to compile whereas it should.
For 2 I can see the value of the signal. This is what Cargo.lock is for, specifying the versions of the deps that are known to work, so I find it a bit out of place in the TOML but OK. However I don't understand transitivity here—if my dep hasn't been tested on Windows, and I have tested my crate on Windows and it works, then should it matter that the dep isn't tested?
Finally, if we don't specify whether it's 1 or 2 and leave it to package maintainers then we can get in the glorious situation where a cargo update bumps two deps an now we stop compiling on platforms where really we should (because someone didn't test it yet on that platform and that maintainer used definition 2) and we don't work on platforms where we shouldn't even try (because someone marked it as compiling and the maintainer used definition 1).
All of this is somewhat mitigated by this check being a warning that can be silenced with an override in Cargo.toml, but again this will bit rot. Are we proposing here a hard failure, a warning, or just adding a lint?
Apologies for the wall of text, just feel super affected by this.
. I don't understand the RFC's section on motivation well enough to understand if this will help me.
There are several motivations. I'll put them in the order of what I feel their importance is
- Skipping unrelated dependencies during
cargo vendor - Being able to run
cargo check --workspacewithout having to exclude a lot of targets - Pruning unrelated dependencies from
Cargo.lock - More targeted error messages
In https://github.com/rust-lang/rfcs/pull/3759#discussion_r1910646224 and https://github.com/rust-lang/rfcs/pull/3759#discussion_r1909379623, I suggested we punt on all of these but (2) because it is the simplest, smallest set of functionality we can get and focusing the conversation on such a subset will likely make the conversation here easier and the whole process for all of the features faster.
I also don't understand what supported-platforms would signal. I see two options.
This code compiles on the following platforms. This code has been verified to work on the following platforms.
This is talked about in many points in the RFC already. I think many of us think it should not be about what is verified and is one of the reasons I suggest moving away from the word "support" in https://github.com/rust-lang/rfcs/pull/3759#discussion_r1909408062
We have never had a crate not compile on a system and not understand why just by looking at the crate's name. We have never had a crate compile but work incorrectly on a system. I don't understand the RFC's section on motivation well enough to understand if this will help me.
...
For 1 if we're using it basically to cache the results of compilation the obvious issue is running out of sync. If we're not compiling because of a dependency and they fix the dependency, now our package won't even attempt to compile whereas it should.
The error message would be less useful in you situation and more helpful for libraries on crates.io where what is needed to build the library is less obvious and you might not find out until you either get an error message about a function not existing or get a compile_error!
And this isn't about caching where the result would be invalidated by a code fix but about declaring the requirements needed to build the package.
I have updated the RFC to address what was discussed, namely:
- I decided to follow the advice of treating one topic at a time. So, dependency pruning & errors are now moved to future possibilities, the RFC now only covers build-target filtering in workspaces.
- The field is now a single string which is a
cfg(...)specification, following the discussion here.
@joshtriplett I believe both of your concerns are no longer in scope, since the compatibility of a dependency with supported-targets is not checked, and the field is now removed upon publishing (local only).
Concerns that remain unresolved:
- The name of the field
- The format of the field
2. Being able to run
cargo check --workspacewithout having to exclude a lot of targets
Thank you for taking the time to write this thoughtful explanation, I find the RFC easier to understand now. And, in fact, solves a real problem we have. I would love to be able to do
[workspace.'cfg(not(target_os = "linux"))'.exclude]
and not have to resort to our current shenanigans. We currently have binaries with shims like:
#[cfg(not(target_os = "linux"))]
fn main() {
eprintln("Not supported on this platform");
}
...and libraries that compile to nothing on wrong platforms.
However, I think that this is OK because it buys us capability-based checks vs. version-based checks. This is a big deal for me. For a flowery exposition, read Linus Torvald's posts about x86 feature levels. In our own code, the lack of platform checks has been a forcing function which caused us to have separate features for e.g. zbus, rpi-gpio and termios2 whereas if we had the hammer of cfg(target_os = "linux") we probably would have used that and gotten brittle, non-future-proof code. For example, zbus works on macOS and if we needed to use it for some tests and didn't control the crate, with the proposed solution we would have probably been out of luck. So in a way I feel this has a potential to subvert cargo features and this is something we can't fix with a flag.
I'm also marginally concerned about the quality of the user-supplied data here. Once a crate works for the library author for their platform, they have little incentives to add other platforms. My gut feeling is that this will make embedded libraries harder to use in the long run, specially for people who are adding new platforms.