proc_macro should be in the prelude

[nagisa]

As of https://github.com/rust-lang/cargo/pull/7700 --extern proc_macro is getting added to the prelude of the rustc invocations cargo makes when building proc_macro crates.

As crate2nix/buildRustCrate (not sure where exactly this would go, filling this issue out without investigating too much) does not add this flag, there’s some discrepancy between how rustc behaves when invoked by cargo and by crate2nix. In particular cargo does not require extern crate proc_macro anymore to use proc_macro in proc_macro crates, so eventually things will start failing to compile when this gets to stable and crates stop including the extern crate.

[nagisa]

The change is now in 1.42.0:

https://blog.rust-lang.org/2020/03/12/Rust-1.42.html#use-proc_macro:tokenstream;-now-works

[kolloch]

A failing minimal example would be appreciated.

[nagisa]

$ cat Cargo.toml
[package]
name = "foo"
version = "0.1.0"
authors = ["Simonas Kazlauskas <[email protected]>"]
edition = "2018"

[lib]
proc-macro=true

[dependencies]
$ cat src/lib.rs
use proc_macro::TokenTree;
$ cargo build
warning: unused import: `proc_macro::TokenTree`
 --> src/lib.rs:1:5
  |
1 | use proc_macro::TokenTree;
  |     ^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
$ crate2nix generate -o Cargo.nix
Generated Cargo.nix successfully.
$ nix build -f Cargo.nix
trace: root_crate is deprecated since crate2nix 0.4. Please use rootCrate instead.
trace: workspace_members is deprecated in crate2nix 0.4. Please use workspaceMembers instead.
builder for '/nix/store/0ihm9ssixkaim9wy09drb33kwxjdsfmd-rust_foo-0.1.0.drv' failed with exit code 1; last 10 log lines:
  Running rustc --crate-name foo src/lib.rs --out-dir target/lib --emit=dep-info,link -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=16 --remap-path-prefix=/build=/ --cfg feature="default" --edition 2018 -C metadata=7bded8e7cd -C extra-filename=-7bded8e7cd --crate-type proc-macro --color always
  error[E0432]: unresolved import `proc_macro`
   --> src/lib.rs:1:5
    |
  1 | use proc_macro::TokenTree;
    |     ^^^^^^^^^^ use of undeclared type or module `proc_macro`

  error: aborting due to previous error

  For more information about this error, try `rustc --explain E0432`.
[0 built (1 failed)]
error: build of '/nix/store/0ihm9ssixkaim9wy09drb33kwxjdsfmd-rust_foo-0.1.0.drv' failed

[andir]

cargo addes --extern proc_macro whenever the crate-type is proc-macro:

https://github.com/rust-lang/cargo/commit/4d64eb99a4#diff-7f98585dbf9d30aa100c8318e2c77e79R1021-R1022

This should probably be handled in nixpkgs which I'll prepare a PR for.

[andir]

My main concern, why I actually commented, was why a language feature is actually implemented in cargo and not in rustc. I had the impression they were trying to keep the boundaries somewhat properly. I fear changes like these will cause a lot of churn on the downstream (e.g. nix based, bazel, cmake?, …) tooling for rust as some feature aren't really of the language but of the build tooling. I have to read more into the whole rust(c) development story to figure out if there is a doc specifying the boundaries.

[nagisa]

Technically Cargo.toml and everything related to it is entirely within the world of cargo and cargo deciding to add a flag based on a specification in Cargo.toml does not make it a language feature at all.

In fact, cargo still doesn’t do the same if instead of proc-macro = true you specify crate-type = ["proc-macro"].

OTOH I do sympathize with your concern as pretty much all rust code assumes cargo.

[andir]

I have a prototype for this: https://github.com/andir/nixpkgs/commit/898fb007fc8e725f43028e8cecd4dcfd58320c66

It can only go into nixpkgs once 1.42 has landed in master. It is currently going through the staging cycle so that will only be a day or two more.

[andir]

So, adding --extern proc_macro is apparently not enough:

$ nix-build -A buildRustCrateTests.tests.procMacroInPrelude
trace: { authors = <CODE>; crateName = "procMacroInPrelude"; procMacro = true; src = <CODE>; version = "0.1.0"; }
these derivations will be built:
  /nix/store/0f6ycf3ghxmvy94hz7cbcp2z4226a040-rust_procMacroInPrelude-0.1.0.drv
  /nix/store/m8axk0gy537ka4kxixjfkmiyzfk3a44f-run-buildRustCrate-procMacroInPrelude-test.drv
building '/nix/store/0f6ycf3ghxmvy94hz7cbcp2z4226a040-rust_procMacroInPrelude-0.1.0.drv'...
unpacking sources
unpacking source archive /nix/store/qcd3ij4288xad3i8xl5yxhjqr65vvpg0-proc-macro-in-prelude
source root is proc-macro-in-prelude
patching sources
configuring
Running cd .
building
Building src/lib.rs (procMacroInPrelude)
Running rustc --crate-name procMacroInPrelude src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=24 --remap-path-prefix=/build=/ --extern proc_macro -C metadata=526f22e779 -C extra-filename=-526f22e779 --crate-type proc-macro --color always
error[E0432]: unresolved import `proc_macro`
 --> src/lib.rs:1:5
  |
1 | use proc_macro::TokenTree;
  |     ^^^^^^^^^^ maybe a missing crate `proc_macro`?

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.

Rustc is 1.42.0 in this case.

[nagisa]

@andir you need edition="2018" in your Cargo.toml (or --edition=2018 in your command line is missing) I think. Only since 2018 you may omit extern crate items in source code and that’s where --extern proc-macro starts mattering.