cargo icon indicating copy to clipboard operation
cargo copied to clipboard

Allow specifying a set of supported target platforms in Cargo.toml

Open luser opened this issue 7 years ago • 14 comments

Spun off from the specific proposal in https://github.com/rust-lang/cargo/issues/4544#issuecomment-357371733 "A random solution which may help solve this though is to perhaps list the valid targets for a project in a workspace root Cargo.toml. That way Cargo could be smarter and just vendor dependencies for those targets (and alter resolution so it won't include winapi in the lock file)"

This would help projects like Firefox and Fuchsia that vendor their dependencies by allowing them to avoid vendoring crates that are only necessary for platforms that the project does not support (see also #7058).

cc @cramertj

See also

  • #12260 for specifying build target configuration per platform
  • #9208 for specifying supported platform targets for build targets

luser avatar Oct 16 '18 14:10 luser

This seems plausible to me! Cargo has to execute rustc to learn about cfg information for targets, so that may be one roadblock to getting this working but otherwise I think it in theory shouldn't be too too hard to plumb through what we've got today

alexcrichton avatar Oct 17 '18 15:10 alexcrichton

Similarly, it seems like you could also plumb through the set of features with which each library will be built, so that there's no need to pull in optional dependencies.

cramertj avatar Oct 17 '18 16:10 cramertj

I poked around that a bit and that already seems to work. If you have a dependency on crate A and A has an optional dependency on B, you won't wind up with B in your Cargo.lock unless you enable that feature.

luser avatar Oct 17 '18 17:10 luser

cc @Eh2406, we were just talking about this!

We talked a bit in person about this and envisioned something like:

  • Each workspace has a default set of targets and a default set of features used to generate a lock file. These both default to "everything"
  • You can configure a [workspace] to either change the default set of features or the default set of targets. This affects lock file resolution. It also would be documented with lots of warnings about the effects of doing this (like thrashing lock files on builds)
  • Finally, --minimal-cargo-lock would be a CLI flag to say "ignore configuration, do the minimal thing"

We've already basically got the support for all of this, it'd just need some wiring up and stabilization!

alexcrichton avatar Oct 20 '18 03:10 alexcrichton

I had another use-case for this #8645

Problem: It is difficult to recognize what targets a crate is compatible with. With WASM coming about, it has become more important to know if a crate is WASM-compatible or not, i.e. builds and runs on a wasm32 architecture. One has to try building the crate for the target manually, and if there are issues one needs to find the dependency which may be causing incompatibility by recursing down the dependency tree, following the compiler errors.

Solution:

[build]
targets = [
    "wasm32-unknown-unknown",
    "wasm32-wasi",
    "x86_64-unknown-linux-gnu",
    "i586-pc-windows-msvc",
]
  • Allow crate authors to define compatible targets in the crate configuration
  • check those targets when publishing the crate
  • let cargo make a suggestion to add a target to the list, after target has been checked successfully
  • check the compatible targets of dependencies and point out compatibility conflicts

This would make the compatibility visible in the crate, and make it easier to find compatibility issues. Another idea is to show the compatible targets in the documentation on docs.rs, to make it even easier to see what targets a crate can be run on.

@alexcrichton I don't understand why this should be bound to workspaces though? Wouldn't this make sense for any crate?

mankinskin avatar Aug 24 '20 14:08 mankinskin

Another idea is to declare something like non-targets to explicitly exclude targets from compatibility for a crate. This could for example be useful when you never intend a crate to target something special like Wasm.

mankinskin avatar Aug 24 '20 14:08 mankinskin

This would help a problem we have in Firefox, where some of the dependencies may have WASM-specific dependencies (https://github.com/grovesNL/glow), which are dragged into the 3rd party sources by cargo vendor.

kvark avatar Aug 19 '21 18:08 kvark

I've just recently hit on a circumstance where this functionality would be very useful: preventing cargo (and vendoring by extension) from pulling in crates that will never be built on the intended target.

Or at least that would be my hope.

An example is that I build and maintain a particular project, and this project has many dependencies. Various of these dependencies have winapi as a dependency themselves. This in itself is not an issue really, as the winapi crate is behind a target.cfg flag, so it never gets built on Linux or Macos.

However! winapi and its own chain of dependencies pulls in close to 350MB of stuff for Windows. Stuff that will never ever be built (on my target). The side effect of this is that when I do vendoring and compress the artifacts, the result with this is a 17MB archive - if I remove the windows crates, it's 3.5MB. It has an impact all the way down the distribution chain where a Linux distro might be using vendoring, and now has to account for 13.5MB of wasted space.

My hope for this issue is that by specifying a a target that the crate is for it would prevent the above scenario by never resolving the windows target. With a default of no target specified defaulting back to current behaviour.

flukejones avatar Dec 19 '21 21:12 flukejones

Just came here from #5220, which was closed in favor of this issue.

Do the supported targets need to be workspace-wide (as suggested by the initial comment here), or could they apply crate-by-crate? Currently e.g. https://github.com/sportsball-ai/av-rs has one workspace with some crates that are portable, some that are macOS-only, and some that are Linux-only. I was following #5220 because I was hoping to be able to just have cargo build do the right thing on the workspace. That seems possible via #6179 if the intended state is you can set supported targets for each crate, but not if you can only do it at the workspace level.

scottlamb avatar Sep 26 '23 18:09 scottlamb

My expectation is we'd have something like

[package]
name = "cargo-credential-macos-keychain"
# ...
required-targets = [
  "aarch64-apple-darwin",
  "aarch64-apple-ios",
  "aarch64-apple-ios-sim",
  "x86_64-apple-darwin",
  "x86_64-apple-ios",
]
  • Ideally we put this in the Index Summary and you can't depend on a package with required-targets unless your required-targets is the same or a superset or you use it in a target-specific dependency
  • I chose "required" in the name to be similar to bin.required-features which means "if these features aren't active, skip this bin".
  • I left off cfg in this initial example
    • I suspect there are corner cases that could lead to Cargo.lock being modified just by building on another system (newer rustc with a new target and a dependency already supports that target with a conditional dependency)
    • I suspect this gets even more complicated if we try to find the intersection between cfg in package.required-targets and in a target dependency
  • We'd need to figure out what validation can or should be done on the target list (maybe just a lint?)
  • I question whether workspace inheritance would be useful but then it might just be good to do just for consistency

epage avatar Sep 26 '23 18:09 epage

Please include some escape hatch that lets you ignore a requirement in a crate you depend on if it's incorrectly too strict.

I expect people will get this wrong with the design suggested above which uses a full target triple by failing to enumerate every possible triple it works on. Something like an option to exclude all windows targets without then enumerating all unixy targets would encourage people to restrict correctly/allow for future new target triples to work.

LunNova avatar Sep 26 '23 22:09 LunNova

Something like an option to exclude all windows targets without then enumerating all unixy targets would encourage people to restrict correctly/allow for future new target triples to work.

I think the exclude-targets option may well be best. Currently there are two scenarios for myself where I might want this:

  • exclude arm-v7 and below
  • exclude windows OS

Perhaps it would be wise to allow wild-carding of parts of a triple like *-apple-darwin, aarch64-*?

flukejones avatar Sep 29 '23 09:09 flukejones

btw there is #9208 for doing this on the build-target / crate level

epage avatar Oct 19 '23 02:10 epage

We open sourced a tool that allows you to specify the supported targets of a crate in the Cargo.toml named cargo-ft. Unfortunately, it doesn't solve the Cargo.lock problem which is probably only reasonably doable in cargo directly. Nevertheless, it simplifies building, checking and testing your code when your workspace consist of multiple binary crates with various target platform support. Hope it can help at least a few other people than us.

stormshield-guillaumed avatar Dec 08 '23 14:12 stormshield-guillaumed

Perhaps it would be wise to allow wild-carding of parts of a triple like -apple-darwin, aarch64-?

Ergonomically specifying that a crate is Wasm-incompatible would require both this and exclude-targets, to allow for something like exclude-targets = [ "wasm32-*" ], as positive lists would be very hard to maintain (and often end up incorrect).

That feature would be most welcome since Wasm-incompatible crates often build but fail at runtime in ways that aren't obviously a platform incompatibility — in the case of typetag, for example, one has to dig down the errors they get in the runtime logs until they find discussions that ultimately mention that the crate doesn't actually work on Wasm.

hishamhm avatar Mar 06 '24 19:03 hishamhm