cargo
cargo copied to clipboard
rust-lang
"can't find crate for..." proc-macro crates when the crate is clearly present and is passed to rustc
Problem
When compiling some crates (poem 3.1.10, salvo 0.78.0), I expect them to build; they fail to build with errors like:
error[E0463]: can't find crate for `salvo_macros`
--> /home/nora/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/salvo_core-0.78.0/src/lib.rs:31:9
|
31 | pub use salvo_macros::handler;
| ^^^^^^^^^^^^ can't find crate
error[E0463]: can't find crate for `salvo_macros`
--> /home/nora/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/salvo_core-0.78.0/src/lib.rs:33:9
|
33 | pub use salvo_macros as macros;
| ^^^^^^^^^^^^ can't find crate
error[E0463]: can't find crate for `salvo_macros`
--> /home/nora/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/salvo_core-0.78.0/src/lib.rs:85:13
|
85 | pub use salvo_macros::{Extractible, handler};
| ^^^^^^^^^^^^ can't find crate
For more information about this error, try `rustc --explain E0463`.
error: could not compile `salvo_core` (lib) due to 3 previous errors
Steps
cargo new whatever && cd whatevercargo add poem(orsalvo)cargo build
However, I tried this on a new VM and wasn't able to reproduce it. It's something to do with my system, and I haven't yet figured out what.
Possible Solution(s)
I tried uninstalling (rustup self uninstall) and reinstalling Rust, as well as cargo clean and rm -r ~/.cargo/registry/cache.
This appears not to happen under rust 1.80 but does happen under 1.81.
Notes
I ran cargo build --verbose and can see that the "missing" crates are indeed being passed to rustc:
/home/nora/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc
--crate-name salvo_core
--edition=2024 /home/nora/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/salvo_core-0.78.0/src/lib.rs
# ...
--extern salvo_macros=/tmp/salvotest/target/debug/deps/libsalvo_macros-2d16cc326ba6665d.so
# ...
--cap-lints allow`
Version
cargo 1.86.0 (adf9b6ad1 2025-02-28)
release: 1.86.0
commit-hash: adf9b6ad14cfa10ff680d5806741a144f7163698
commit-date: 2025-02-28
host: x86_64-unknown-linux-gnu
libgit2: 1.9.0 (sys:0.20.0 vendored)
libcurl: 8.12.0-DEV (sys:0.4.79+curl-8.12.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w 11 Sep 2023
os: Debian 12.0.0 (bookworm) [64-bit]
nora ran cargo bisect-rustc and found this:
Regression in nightly-2024-07-13
...
looking for regression commit between 2024-07-12 and 2024-07-13
fetching (via remote github) commits from max(5315cbe15b79533f380bbb6685aa5480d5ff4ef5, 2024-07-10) to c6727fc9b5c64cefa7263486497ee95e529bd0f8
ending github query because we found starting sha: 5315cbe15b79533f380bbb6685aa5480d5ff4ef5
get_commits_between returning commits, len: 8
commit[0] 2024-07-11: Auto merge of #127609 - flip1995:clippy-subtree-update, r=Manishearth
commit[1] 2024-07-11: Auto merge of #127614 - matthiaskrgr:rollup-8geziwi, r=matthiaskrgr
commit[2] 2024-07-12: Auto merge of #127382 - estebank:const-let, r=compiler-errors
commit[3] 2024-07-12: Auto merge of #127635 - matthiaskrgr:rollup-foopajr, r=matthiaskrgr
commit[4] 2024-07-12: Auto merge of #127479 - Urgau:rustc-stable-hash, r=michaelwoerister
commit[5] 2024-07-12: Auto merge of #127653 - matthiaskrgr:rollup-72bqgvp, r=matthiaskrgr
commit[6] 2024-07-12: Auto merge of #127636 - nnethercote:fix-Parser-look_ahead, r=oli-obk
commit[7] 2024-07-12: Auto merge of #123351 - beetrees:x86-ret-snan-rust, r=nikic,workingjubilee
so a) this is a rustc bug, and b) i suspect it is related to rustc-stable-hash somehow.
we also ran RUSTC_LOG=rustc_metadata=info and found this mismatch between the working and non-working builds:
# working
INFO rustc_metadata::creader resolving crate `poem_derive`
INFO rustc_metadata::creader library for `poem_derive` was loaded previously
# non-working
INFO rustc_metadata::creader resolving crate `poem_derive`
INFO rustc_metadata::creader falling back to a load
INFO rustc_metadata::locator dylib reading metadata from: /tmp/whatever/target/debug/deps/libpoem_derive-a4fba4f212f1e51d.so
INFO rustc_metadata::locator Rejecting via proc macro: expected false got true
INFO rustc_metadata::locator metadata mismatch
INFO rustc_metadata::creader falling back to loading proc_macro
INFO rustc_metadata::locator dylib reading metadata from: /tmp/whatever/target/debug/deps/libpoem_derive-a4fba4f212f1e51d.so
INFO rustc_metadata::creader register newly loaded library for `poem_derive`
INFO rustc_metadata::creader register crate `poem_derive` (cnum = 84. private_dep = false)
this bit here: https://github.com/rust-lang/rust/blob/c9f690a1e3d42f80788e9e4f2aa25547b1d8df58/compiler/rustc_metadata/src/creader.rs#L516-L535 is rejecting a previously loaded crate, but i'm not sure why
there is additional logging in the non-working case that i didn't post at first; here it is:
INFO rustc_metadata::creader resolving crate `poem_derive`
INFO rustc_metadata::creader falling back to a load
INFO rustc_metadata::locator dylib reading metadata from: /tmp/whatever/target/debug/deps/libpoem_derive-a4fba4f212f1e51d.so
INFO rustc_metadata::locator Rejecting via proc macro: expected false got true
INFO rustc_metadata::locator metadata mismatch
INFO rustc_metadata::creader falling back to loading proc_macro
INFO rustc_metadata::locator dylib reading metadata from: /tmp/whatever/target/debug/deps/libpoem_derive-a4fba4f212f1e51d.so
INFO rustc_metadata::creader register newly loaded library for `poem_derive`
INFO rustc_metadata::creader resolving crate `poem_derive`
in particular note those last two lines. "register newly loaded library" should always be followed by "register crate"; the only way it can go wrong is if intern_stable_crate_id returns an error.
my working theory:
existing_match, for some reason, incorrectly thinks it needs to reload the crate because of a mismatch.CrateLocatorsuccessfully finds the crate and passes it toregister_crateregister_cratefails to reregister it, because it is the same crate and it rejects duplicate StableCrateIds.
cc @petrochenkov @Urgau i suspect this is related to https://github.com/rust-lang/rust/pull/127479 somehow but i'm not sure how. do you have ideas on what could be going wrong in existing_match?
i think there is a secondary issue which is that this should have reported E0519 instead of "crate not found". the last code to touch that was https://github.com/rust-lang/rust/pull/111461 but that says it was just readding existing logic.
ok yeah https://github.com/rust-lang/rust/pull/109213/ really should not have removed the bug!; cc @oli-obk
I re-examine https://github.com/rust-lang/rust/pull/127479 and I don't see anything that would explain the behavior seen here.
I also re-compared the extracted code with the code inside rustc-stable-hash and didn't find any meaningful difference.
@jyn514 Are you able to confirm that the crate hash are not the same? Does reverting the PR fixes the issue?
@Urgau i'm not able to replicate this locally, so i can't confirm anything about the hashes one way or another. but i don't see anything else in the range bisect-rustc shows that could possible affect metadata loading. and the error seems related to hashing somehow, because the issue is that rustc_resolve thinks the .so file doesn't match the already loaded cnum, even though later it errors because file has the same StableCrateId.
I tried locally and on a VM and I'm also unable to reproduce it, with both poem and salvo.
Do you have the Nix package manager installed? I am using Nix on Ubuntu and ran into a similar issue with wasmtime. Removing gcc that was installed via the Nix Home Manager solved it for me. Someone else on the Rust user forums seems to have encountered the same problem.
I bumped into this a couple of times with nixpkgs but never remembered how I fixed it. Guess it might be nix patch-elf at sometimes patching rustc wrongly, or my rust toolchain frkm nixpkgs was missing some dependencies.
if someone encountering this problem could run nm -g target/debug/deps/*.so | grep __rustc_proc_macro_decls_ and send me the output, that would be very helpful. if that has no output, send the output without the pipe to grep.
$ nm -g target/debug/deps/*.so | grep __rustc_proc_macro_decls_
00000000001ed5e8 D __rustc_proc_macro_decls_43a8767dcd1f163b__
000000000033d008 D __rustc_proc_macro_decls_9ad6036aa6c77638__
0000000000308618 D __rustc_proc_macro_decls_2ed9d063c2b4b61a__
00000000002676a0 D __rustc_proc_macro_decls_2ed462a267df31a2__
00000000002178b0 D __rustc_proc_macro_decls_f3fa41c5b5ca6687__
0000000000288908 D __rustc_proc_macro_decls_7f632681e18fe3f7__
Removing the home-manager managed gcc did not help; I am using a Rustup not installed through Nix.
I've tried to reproduce it and wasn't able to. It would be useful to get more details on the OS setup:
- What distro? NixOS or a different distro with Nix installed or something different entirely?
- Where was Rust installed from?
- What sort of relevant toolchains are present and from where?
Apologies for the long delay.
What distro? NixOS or a different distro with Nix installed or something different entirely?
Debian GNU/Linux 12 (bookworm) x86_64 with Nix managing packages via home-manager.
Where was Rust installed from?
rustup.rs
What sort of relevant toolchains are present and from where?
Rust toolchains installed:
~
❯ rustup show
Default host: x86_64-unknown-linux-gnu
rustup home: /home/nora/.rustup
installed toolchains
--------------------
stable-x86_64-unknown-linux-gnu (active, default)
nightly-x86_64-unknown-linux-gnu
1.80.0-x86_64-unknown-linux-gnu
1.81.0-x86_64-unknown-linux-gnu
1.85.0-x86_64-unknown-linux-gnu
stage1
active toolchain
----------------
name: stable-x86_64-unknown-linux-gnu
active because: it's the default toolchain
installed targets:
x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl
C toolchains installed:
~
❯ apt-cache policy gcc
gcc:
Installed: 4:12.2.0-3
Candidate: 4:12.2.0-3
Version table:
*** 4:12.2.0-3 500
500 http://debian.uchicago.edu/debian bookworm/main amd64 Packages
100 /var/lib/dpkg/status
~
❯ apt-cache policy clang
clang:
Installed: 1:14.0-55.7~deb12u1
Candidate: 1:14.0-55.7~deb12u1
Version table:
*** 1:14.0-55.7~deb12u1 500
500 http://debian.uchicago.edu/debian bookworm/main amd64 Packages
100 /var/lib/dpkg/status
home-manager stuff installed:
``` ~ ❯ home-manager packages act-0.2.71 atuin-18.4.0 bat-0.25.0 bottom-0.10.2 broot-1.44.6 chafa-1.14.5 ctpv-1.1 cyme-1.8.5 delta-0.18.2 deno-2.1.7 dfc-3.1.1 direnv-2.35.0 dua-2.29.4 exercism-3.5.4 eza-0.20.18 eza-0.20.18-man fd-10.2.0 fend-1.5.5 fortune-kind-custom freshfetch-0.2.0 fzf-0.58.0 fzf-0.58.0-man git-2.47.1 go-1.23.4 graphviz-12.2.1 hexyl-0.16.0 hm-session-vars.sh home-configuration-reference-manpage home-manager htop-3.3.0 iotop-0.6 jq-1.7.1-bin jq-1.7.1-man jujutsu-0.25.0 just-1.39.0 just-1.39.0-man man-db-2.13.0 monolith-2.8.3 neocities-0.0.18 neofetch-unstable-2021-12-10 neofetch-unstable-2021-12-10-man neovim-0.10.3 nix-output-monitor-2.1.4 nix-zsh-completions-0.5.1 npins-0.2.4 pandoc-cli-3.1.11.1 procs-0.14.9 ripgrep-14.1.1 starship-1.22.1 syncthing-1.29.2 timg-1.6.1 tor-browser-14.0.4 uutils-coreutils-0.0.29 uv-0.3.0 viddy-1.3.0 wget-1.25.0 wl-clipboard-2.2.1 xclip-0.13 xsel-1.2.1 yt-dlp-2025.1.26 zellij-0.41.2 zsh-5.9 zsh-5.9-man ```
EDIT: This was a user error, I'd asked and no one noticed it was missing -L. But this does at least show the error message has become worse:
Previous contents
Trying to compile some crates with rustc directly, I encountered this error.
Here is my MWE: https://github.com/midnightveil/rustc-cant-find-crate. The RUSTC_LOG="debug" log is similar as the above outputs. Interestingly enough even attaching -C metadata=<random> makes no difference.
mkdir -p build/
rustc --target aarch64-unknown-none --edition 2021 \
--emit=link=build/libsecond.rlib \
--emit=metadata=build/libsecond.rmeta \
--crate-type rlib \
--crate-name second \
second.rs
rustc --target aarch64-unknown-none --edition 2021 \
--emit=link=build/libfirst.rlib \
--emit=metadata=build/libfirst.rmeta \
--crate-type rlib \
--crate-name first \
--extern second=build/libsecond.rlib \
first.rs
rustc --target aarch64-unknown-none --edition 2021 \
--emit=link=build/main.a \
--crate-type staticlib \
--crate-name main \
--extern first=build/libfirst.rlib \
main.rs
error[E0463]: can't find crate for `first`
--> main.rs:6:5
|
6 | use first;
| ^^^^^ can't find crate
error: aborting due to 1 previous error
For more information about this error, try `rustc --explain E0463`.
make: *** [Makefile:6: build/main.a] Error 1
On an older version of rustc, this error occurs, which doesn't make much sense to me either:
make RUSTC="rustc +1.70.0"
mkdir -p build/
rustc +1.70.0 --target aarch64-unknown-none --edition 2021 \
--emit=link=build/libsecond.rlib \
--emit=metadata=build/libsecond.rmeta \
--crate-type rlib \
--crate-name second \
second.rs
rustc +1.70.0 --target aarch64-unknown-none --edition 2021 \
--emit=link=build/libfirst.rlib \
--emit=metadata=build/libfirst.rmeta \
--crate-type rlib \
--crate-name first \
--extern second=build/libsecond.rlib \
first.rs
rustc +1.70.0 --target aarch64-unknown-none --edition 2021 \
--emit=link=build/main.a \
--crate-type staticlib \
--crate-name main \
--extern first=build/libfirst.rlib \
main.rs
error[E0519]: the current crate is indistinguishable from one of its dependencies: it has the same crate-name `first` and was compiled with the same `-C metadata` arguments. This will result in symbol conflicts between the two.
--> main.rs:6:5
|
6 | use first;
| ^^^^^
error: aborting due to previous error
For more information about this error, try `rustc --explain E0519`.
make: *** [Makefile:6: build/main.a] Error 1
I can reproduce this both with and without nix.
(There's a couple more comments in the repo: for one, adding both crates as deps of the main crate doens't make a difference, and also using extern crate first; instead of use first; tells us that it's complaining about not being able to find the dependency second or first)
Relevant-ish metadata
I use Fedora, with rustup installed via nix home-manager. I can also reproduce this on a debian system with rustc installed via rustup.
> rustc --version
rustc 1.87.0 (17067e9ac 2025-05-09)
> which rustc
/home/julia/.nix-profile/bin/rustc
> rustup show
Default host: x86_64-unknown-linux-gnu
rustup home: /home/julia/.rustup
installed toolchains
--------------------
stable-x86_64-unknown-linux-gnu (default)
nightly-2025-02-26-x86_64-unknown-linux-gnu
nightly-x86_64-unknown-linux-gnu
1.70.0-x86_64-unknown-linux-gnu
1.81.0-x86_64-unknown-linux-gnu
installed targets for active toolchain
--------------------------------------
aarch64-unknown-linux-gnu
aarch64-unknown-none
riscv64gc-unknown-none-elf
x86_64-unknown-linux-gnu
x86_64-unknown-linux-musl
x86_64-unknown-none
active toolchain
----------------
stable-x86_64-unknown-linux-gnu (default)
rustc 1.87.0 (17067e9ac 2025-05-09)
> make
mkdir -p build/
rustc --target aarch64-unknown-none --edition 2021 \
--emit=link=build/libsecond.rlib \
--emit=metadata=build/libsecond.rmeta \
--crate-type rlib \
--crate-name second \
second.rs
rustc --target aarch64-unknown-none --edition 2021 \
--emit=link=build/libfirst.rlib \
--emit=metadata=build/libfirst.rmeta \
--crate-type rlib \
--crate-name first \
--extern second=build/libsecond.rlib \
first.rs
rustc --target aarch64-unknown-none --edition 2021 \
--emit=link=build/main.a \
--crate-type staticlib \
--crate-name main \
--extern first=build/libfirst.rlib \
main.rs
error[E0463]: can't find crate for `first`
--> main.rs:6:5
|
6 | use first;
| ^^^^^ can't find crate
error: aborting due to 1 previous error
For more information about this error, try `rustc --explain E0463`.
make: *** [Makefile:6: build/main.a] Error 1
> which rustup
/home/juliab/.cargo/bin/rustup
> which rustc
/home/juliab/.cargo/bin/rustc
(jyn sent me here 💜)
@midnightveil That's normal. rustc does not implicitly have a search path for transitive dependencies. You'll need to add -L build for it to find them.
