cargo icon indicating copy to clipboard operation
cargo copied to clipboard

`cargo rustdoc --output-format=json` sometimes doesn't rebuild for different versions of same crate name.

Open aDotInTheVoid opened this issue 1 month ago • 6 comments

Problem

When multiple dependencies have the same name, running cargo rustdoc --output-format=json -p <pkg>@<ver> sometimes won't re-run rustdoc if the version has changed.

Steps

(For a repo containing this, see https://codeberg.org/adot/cargo-rustdoc-json-bug).

We have a crate entry that depends both on [email protected] and [email protected]

File: entry/Cargo.toml

[package]
name = "entry"
version = "0.1.0"
edition = "2024"

[dependencies]
dep_v1 = { path = "../dep_v1", package = "dep" }
dep_v2 = { path = "../dep_v2", package = "dep" }

File: dep_v1/Cargo.toml

[package]
name = "dep"
version = "1.0.0"
edition = "2024"

File: dep_v2/Cargo.toml

[package]
name = "dep"
version = "2.0.0"
edition = "2024"

Then in entry/ run:

  1. Make sure we're doing a clean build

    $ cargo clean
     Removed 120 files, 637.0KiB total
    
  2. Build rustdoc-json for [email protected]:

    $ cargo rustdoc --output-format=json -Zunstable-options [email protected]
     Documenting dep v1.0.0 (/home/alona/tmp/cargo-rustdoc-json-bug/dep_v1)
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.11s
       Generated /home/alona/tmp/cargo-rustdoc-json-bug/entry/target/doc/dep.json
    

    This runs rustdoc, as expected.

  3. Build rustdoc-json for [email protected]:

    $ cargo rustdoc --output-format=json -Zunstable-options [email protected]
     Documenting dep v2.0.0 (/home/alona/tmp/cargo-rustdoc-json-bug/dep_v2)
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.56s
       Generated /home/alona/tmp/cargo-rustdoc-json-bug/entry/target/doc/dep.json
    

    Even though it's outputting the same path, in re-run rustdoc, because the package being documented has changed.

  4. Re-build rustdoc-json for [email protected]:

    $ cargo rustdoc --output-format=json -Zunstable-options [email protected]
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s
    Generated /home/alona/tmp/cargo-rustdoc-json-bug/entry/target/doc/dep.json
    

    For some reason this time, it doesn't re-run rustdoc. The file dep.json now contains output for [email protected], not [email protected]

Possible Solution(s)

Either cargo could somehow re-build in step 3 (as well as 2, as it currently does). I'm not familiar enough with cargo's caching logic to say why this happens.

Alternatively we could resolve this in rustdoc, by fixing https://github.com/rust-lang/rust/issues/142370. This would involve rustdoc adding the value of -Cmetadata or -Cextra-filename. I think this is preferable because it means that a user can switch between building [email protected] and [email protected] without having to rebuild each time, because they're not the same.

Notes

No response

Version

cargo 1.93.0-nightly (5c0343317 2025-11-18)
release: 1.93.0-nightly
commit-hash: 5c0343317ce45d2ec17ecf41eaa473a02d73e29c
commit-date: 2025-11-18
host: x86_64-unknown-linux-gnu
libgit2: 1.9.1 (sys:0.20.2 vendored)
libcurl: 8.15.0-DEV (sys:0.4.83+curl-8.15.0 vendored ssl:OpenSSL/3.5.4)
ssl: OpenSSL 3.5.4 30 Sep 2025
os: Ubuntu 25.4.0 (plucky) [64-bit]

aDotInTheVoid avatar Nov 22 '25 18:11 aDotInTheVoid

Away from keyboard right now. Could you help check whether -Zrustdoc-depinfo fixed it?

https://doc.rust-lang.org/cargo/reference/unstable.html#rustdoc-depinfo

weihanglo avatar Nov 22 '25 18:11 weihanglo

Issue is still present with -Zrustdoc-depinfo:

$ pwd
/home/alona/tmp/cargo-rustdoc-json-bug/entry
$ cargo clean
     Removed 0 files
$ cargo rustdoc --output-format=json -Zunstable-options [email protected] -Zrustdoc-depinfo
 Documenting dep v1.0.0 (/home/alona/tmp/cargo-rustdoc-json-bug/dep_v1)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.40s
   Generated /home/alona/tmp/cargo-rustdoc-json-bug/entry/target/doc/dep.json
$ cargo rustdoc --output-format=json -Zunstable-options [email protected] -Zrustdoc-depinfo
 Documenting dep v2.0.0 (/home/alona/tmp/cargo-rustdoc-json-bug/dep_v2)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.59s
   Generated /home/alona/tmp/cargo-rustdoc-json-bug/entry/target/doc/dep.json
$ cargo rustdoc --output-format=json -Zunstable-options [email protected] -Zrustdoc-depinfo
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.01s
   Generated /home/alona/tmp/cargo-rustdoc-json-bug/entry/target/doc/dep.json

aDotInTheVoid avatar Nov 22 '25 19:11 aDotInTheVoid

Cargo compares output file's mtime (target/doc/dep.json) with all its dependency outputs mtime to determine whether it needs to rebuild (see this check_filesystem()).

It is not an issue for cargo check or cargo build because they output files (.rlib, .rmeta, etc.) has hash suffix. The file in the top-level output location are uplifted after. However, the rustdoc JSON format was written directly to target/doc/dep.json not some path like target/debug/deps/dep-abcdef1234.json. Cargo assumed the dep.json was up-to-date so didn't rerun.

weihanglo avatar Nov 23 '25 15:11 weihanglo

Alternatively we could resolve this in rustdoc, by fixing rust-lang/rust#142370. This would involve rustdoc adding the value of -Cmetadata or -Cextra-filename. I think this is preferable because it means that a user can switch between building [email protected] and [email protected] without having to rebuild each time, because they're not the same.

Yeah this may be a preferable solution. Just note that target/doc/ directory is a part of final build artifact directory, which is meant to have no intermediate cache and the file name is better to be predictable. I guess if we're going to add suffix, maybe it should be written to build-dir first, and Cargo can uplift the file there.

Maybe this could work well with https://github.com/rust-lang/rust/issues/130676 and https://github.com/rust-lang/cargo/pull/16167, where the uplift is the final step of a cargo doc invocation.

weihanglo avatar Nov 23 '25 16:11 weihanglo

Let me add this as blocked-external, since t-cargo and t-rustdoc need to agree on some designs here.

@rustbot label +S-blocked-external -S-triage +A-rebuild-detection

weihanglo avatar Nov 23 '25 16:11 weihanglo

See #t-cargo > rustdoc-json output filename design.

Cargo might need to change --out-dir to something location like target/path/to/private/location/foo-12ab34ef/foo.json, and uplift to target/doc/foo.json.

  • We need to make sure that --message-format=json has artifact message showing the private location for programmatic use cases.
  • This is not an ultimate solution of JSON output name collision, as the uplifted location will still have it. However, this is the same as running cargo b -p dep@1 && cargo b -p dep@2 and you have only one target/debug/libdep.rlib.

weihanglo avatar Nov 24 '25 21:11 weihanglo