cargo icon indicating copy to clipboard operation
cargo copied to clipboard

cargo vendor without dev-dependencies?

Open RazrFalcon opened this issue 6 years ago • 6 comments

Is there a way to create a vendored sources without dev-dependencies?

RazrFalcon avatar Jun 23 '19 15:06 RazrFalcon

It looks like cargo vendor correctly ignores dev-dependencies for a crate's first-level dependencies, but any sub-dependencies get their dev ones pulled in.

An example using the chrono crate, which pulls in time, which has dev-dependencies winapi, winapi-i686-pc-windows-gnu, and winapi-x86_64-pc-windows-gnu:

$ cargo vendor
   Vendoring autocfg v1.0.0 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/autocfg-1.0.0) to vendor/autocfg
   Vendoring chrono v0.4.10 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/chrono-0.4.10) to vendor/chrono
   Vendoring libc v0.2.67 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.67) to vendor/libc
   Vendoring num-integer v0.1.42 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/num-integer-0.1.42) to vendor/num-integer
   Vendoring num-traits v0.2.11 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/num-traits-0.2.11) to vendor/num-traits
   Vendoring redox_syscall v0.1.56 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/redox_syscall-0.1.56) to vendor/redox_syscall
   Vendoring time v0.1.42 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/time-0.1.42) to vendor/time
   Vendoring winapi v0.3.8 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-0.3.8) to vendor/winapi
   Vendoring winapi-i686-pc-windows-gnu v0.4.0 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-i686-pc-windows-gnu-0.4.0) to vendor/winapi-i686-pc-windows-gnu
   Vendoring winapi-x86_64-pc-windows-gnu v0.4.0 (/home/proxmox/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-x86_64-pc-windows-gnu-0.4.0) to vendor/winapi-x86_64-pc-windows-gnu
To use vendored sources, add this to your .cargo/config for this project:

[source.crates-io]
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "vendor"

However, if we perform a cargo check without vendor set, then those winapi crates are not pulled in:

$ cargo check
    Updating crates.io index
   Compiling autocfg v1.0.0
   Compiling libc v0.2.67
   Compiling test-things v0.1.0 (/home/proxmox/Projects/test-things)
   Compiling num-traits v0.2.11
   Compiling num-integer v0.1.42
    Checking time v0.1.42
    Checking chrono v0.4.10

    Finished dev [unoptimized + debuginfo] target(s) in 5.16s
$ ls target/debug/deps/
autocfg-fa69ae4854af9d10.d        libautocfg-fa69ae4854af9d10.rmeta  liblibc-641bd962ed50a1c5.rmeta         libtest_things-c8241d727281be73.rmeta  num_integer-e2da41134dcd3031.d  test_things-f348b1360dc4a36a.d
chrono-04891ab65662f7bc.d         libc-641bd962ed50a1c5.d            libnum_integer-e2da41134dcd3031.rmeta  libtest_things-f348b1360dc4a36a.rmeta  num_traits-771bd41382020f10.d   time-97f7d8ef4071ccac.d
libautocfg-fa69ae4854af9d10.rlib  libchrono-04891ab65662f7bc.rmeta   libnum_traits-771bd41382020f10.rmeta   libtime-97f7d8ef4071ccac.rmeta         test_things-c8241d727281be73.d

mal20k avatar Mar 06 '20 21:03 mal20k

I am experiencing the same issue. Is there any hope to prioritize and fix this? It is suboptimal to commit dozens/hundreds of dependency folders that are effectively unused.

benma avatar Sep 21 '21 17:09 benma

It sounds great to save bandwidth by not vendoring any dev-dependencies. However, it will be frustrating when someone tries to run cargo test and fails, since the de facto way to use vendor sources is to replace the whole crates.io source.

It looks like cargo vendor correctly ignores dev-dependencies for a crate's first-level dependencies, but any sub-dependencies get their dev ones pulled in.

I believe this is not true. IIRC, Cargo would vendor

  • all direct dependencies listed in Cargo.toml, including build and dev dependencies, and
  • transitive dependencies that are required to build themselves (e.g. normal and build dependencies).

As a result, dev-dependencies of your dependencies would not be downloaded. The only downloaded dev-dependencies are what you list in Cargo.toml of your package.

weihanglo avatar Dec 05 '21 08:12 weihanglo

I can confirm top level dev dependencies get vendored

cargo init
cargo add --dev rand
cargo vendor
ls vendor

iCodeSometime avatar Oct 06 '22 04:10 iCodeSometime

This is also affecting dependency vendoring on Fuchsia via rules_rust.

djkoloski avatar Nov 17 '22 22:11 djkoloski

To cover different use cases we can add a generic approach similar to the cargo tree option edges.

We can call this new option --dep-kinds (As proposed for cargo metadata in https://github.com/rust-lang/cargo/issues/10718) using the same proposed values:

  • all (default for backwards compatibility)
  • normal
  • build
  • dev
  • no-normal
  • no-build
  • no-dev

piotr-sk avatar Oct 09 '24 15:10 piotr-sk

For those seeking a solution to this problem until the issue is resolved, consider using cargo-vendor-filterer.

You can exclude development crates by utilizing the --keep-dep-kinds=no-dev flag.

Please be aware that this solution is not perfect, as it initially vendors all folders and subsequently removes the development crates' code. An optimal solution will be implemented upon the completion of this feature. 😉

piotr-sk avatar Oct 22 '24 05:10 piotr-sk