Duplicated sources when using a workspace (multi-crates) repository

[hectorj]

Hi!

I'm having some trouble when using crates from a repository which defines a workspace (multiple crates).

Here are the relevant bits of my example:

Cargo.toml

[dependencies]
virtual-dom-rs = { git = "https://github.com/chinedufn/percy", rev="c5798a9f46719cf358c58b934110a59706c00f4d" }

Cargo.lock

[[package]]
name = "html-macro"
version = "0.1.9"
source = "git+https://github.com/chinedufn/percy?rev=c5798a9f46719cf358c58b934110a59706c00f4d#c5798a9f46719cf358c58b934110a59706c00f4d"
dependencies = [
 "html-validation",
 "proc-macro2 0.4.30",
 "quote 0.6.13",
 "syn 0.15.44",
]

[[package]]
name = "html-validation"
version = "0.1.2"
source = "git+https://github.com/chinedufn/percy?rev=c5798a9f46719cf358c58b934110a59706c00f4d#c5798a9f46719cf358c58b934110a59706c00f4d"
dependencies = [
 "lazy_static",
]

[[package]]
name = "virtual-dom-rs"
version = "0.6.14"
source = "git+https://github.com/chinedufn/percy?rev=c5798a9f46719cf358c58b934110a59706c00f4d#c5798a9f46719cf358c58b934110a59706c00f4d"
dependencies = [
 "html-macro",
 "js-sys",
 "virtual-node",
 "wasm-bindgen",
 "web-sys",
]

[[package]]
name = "virtual-node"
version = "0.2.7"
source = "git+https://github.com/chinedufn/percy?rev=c5798a9f46719cf358c58b934110a59706c00f4d#c5798a9f46719cf358c58b934110a59706c00f4d"
dependencies = [
 "html-validation",
 "js-sys",
 "lazy_static",
 "wasm-bindgen",
 "web-sys",
]

The error:

++ crate2nix generate -f ./Cargo.toml -o Cargo-generated.nix -h /nix/store/[...]-crate2nix/crate-hashes.json
Error: while retrieving metadata about ./Cargo.toml: Error during execution of `cargo metadata`: error: could not load Cargo configuration

Caused by:
  could not parse TOML configuration in `/nix/store/[...]-crate2nix/cargo/config`

Caused by:
  could not parse input as TOML

Caused by:
  redefinition of table `source.https://github.com/chinedufn/percy` for key `source.https://github.com/chinedufn/percy` at line 10 column 1

and the cargo/config file referenced in the error:

[source.crates-io]
replace-with = "vendored-sources"

[source."https://github.com/chinedufn/percy"]
git = "https://github.com/chinedufn/percy"
rev = "c5798a9f46719cf358c58b934110a59706c00f4d"

replace-with = "vendored-sources"

[source."https://github.com/chinedufn/percy"]
git = "https://github.com/chinedufn/percy"
rev = "c5798a9f46719cf358c58b934110a59706c00f4d"

replace-with = "vendored-sources"

[source."https://github.com/chinedufn/percy"]
git = "https://github.com/chinedufn/percy"
rev = "c5798a9f46719cf358c58b934110a59706c00f4d"

replace-with = "vendored-sources"

[source."https://github.com/chinedufn/percy"]
git = "https://github.com/chinedufn/percy"
rev = "c5798a9f46719cf358c58b934110a59706c00f4d"

replace-with = "vendored-sources"


[source.vendored-sources]
directory = "/nix/store/zl9cqslk607nr8hvbgnpl5jj3bxg5vxv-deps"

It seems like some deduplication is necessary

And here is a direct link to the dependency's Cargo.toml: https://github.com/chinedufn/percy/blob/c5798a9f46719cf358c58b934110a59706c00f4d/Cargo.toml

[kolloch]

Huh! Yes, these should be deduplicated:

https://github.com/kolloch/crate2nix/blob/7902f754b7ce5bc6d4083d7d969e5e3f9506a1c0/tools.nix

Potentially, we also need to have the rev as part of the key, not sure about that.

[pinpox]

Running into the same error while trying to build vaultwarden. @hectorj did you find any workarounds? I'm using this flake.nix and get a very similar error:

@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/7g0x0v8r7n2w8hrh7rlzkbmkccxkzyjd-2w622xckgivjgh127r1xkzyf2d5fqgld-source
source root is 2w622xckgivjgh127r1xkzyf2d5fqgld-source
@nix { "action": "setPhase", "phase": "buildPhase" }
building
++ crate2nix generate -f ./Cargo.toml -o Cargo-generated.nix -h /nix/store/dy8bjd0sf655w9862zj46wbzd44ivj80-vaultwarden-crate2nix/crate-hashes.json
Error: while retrieving metadata about ./Cargo.toml: Error during execution of `cargo metadata`: error: could not load Cargo configuration

Caused by:
  could not parse TOML configuration in `/nix/store/dy8bjd0sf655w9862zj46wbzd44ivj80-vaultwarden-crate2nix/cargo/config`

Caused by:
  could not parse input as TOML

Caused by:
  redefinition of table `source.https://github.com/SergioBenitez/Devise.git` for key `source.https://github.com/SergioBenitez/Devise.git` at line 16 column 1

crate2nix failed.
== cargo/config (BEGIN)
    [source.crates-io]
    replace-with = "vendored-sources"

    [source."https://github.com/servo/rust-url"]
    git = "https://github.com/servo/rust-url"
    rev = "eb7330b5296c0d43816d1346211b74182bb4ae37"

    replace-with = "vendored-sources"

    [source."https://github.com/SergioBenitez/Devise.git"]
    git = "https://github.com/SergioBenitez/Devise.git"
    rev = "e58b3ac9afc3b6ff10a8aaf02a3e768a8f530089"
    branch = "master"
    replace-with = "vendored-sources"

    [source."https://github.com/SergioBenitez/Devise.git"]
    git = "https://github.com/SergioBenitez/Devise.git"
    rev = "e58b3ac9afc3b6ff10a8aaf02a3e768a8f530089"
    branch = "master"
    replace-with = "vendored-sources"

    [source."https://github.com/SergioBenitez/Devise.git"]
    git = "https://github.com/SergioBenitez/Devise.git"
    rev = "e58b3ac9afc3b6ff10a8aaf02a3e768a8f530089"
    branch = "master"
    replace-with = "vendored-sources"

    [source."https://github.com/jjlin/job_scheduler"]
    git = "https://github.com/jjlin/job_scheduler"
    rev = "ee023418dbba2bfe1e30a5fd7d937f9e33739806"

    replace-with = "vendored-sources"

    [source."https://github.com/SergioBenitez/Rocket"]
    git = "https://github.com/SergioBenitez/Rocket"
    rev = "263e39b5b429de1913ce7e3036575a7b4d88b6d7"

    replace-with = "vendored-sources"

    [source."https://github.com/SergioBenitez/Rocket"]
    git = "https://github.com/SergioBenitez/Rocket"
    rev = "263e39b5b429de1913ce7e3036575a7b4d88b6d7"

    replace-with = "vendored-sources"

[litchipi]

@kolloch If you think it's a good first issue, I am willing to try to solve it Are you able point out what is going wrong here, and what is needed to be fixed ? Or should I investigate myself ?

[martyall]

I have also now been bitten by this. It seems like there are some proposed solutions even in PR, and other forks have been using them. Would be great to get a fix in.