Outputting rustdoc JSON output to stdout instead of file when invoked via cargo

[Boscop]

Right now it seems impossible to output the rustdoc JSON output to stdout when it's invoked via cargo, it can only output to a file.

E.g. this produces a JSON file:

cargo +nightly rustdoc -- --output-format json -Z unstable-options

One would expect that this would work to output to stdout

cargo +nightly rustdoc -- --output-format json -Z unstable-options -o -

but unfortunately it doesn't. Instead it gives an error:

error: Option 'out-dir' given more than once

It would be nice if it was possible to output the whole rustdoc JSON output for a crate and all its deps to stdout instead of to a file 🙂

[fmease]

@rustbot transfer cargo

[weihanglo]

Additional flags passed to cargo rustdoc after -- will go to rustdoc invocation. Since Cargo by default sets --out-dir already hence such error.

I haven't tried but --artifact-dir may have some similar effect.

There is an Cargo unstable feature output-format for rustdoc, which takes care of the rebuild detection better than raw cargo rustdoc -- --output-format json. However, it doesn't have a flag to redirect output files either.

whole rustdoc JSON output for a crate and all its deps to stdout instead of to a file

Note that flags specified in cargo rustdoc only applied to the rustdoc invocation of the final artifact, so it is unlikely to take effect on "all its deps".

[weihanglo]

Due to the lack of depinfo files from rustdoc, the current rebuild detection relies on file mtime of the final output HTML/JSON files. It might need a more intrusive design to figure out the support of output to standard stream, and need to consider flag consistency between different commands.

Edit: https://github.com/rust-lang/cargo/issues/15370 addresses this concern.

[weihanglo]

Would you mind sharing why output final JSON files to stdout is important to your workflow, so that we can understand the use case and brainstorm possible solutions?

[Boscop]

@weihanglo

Would you mind sharing why output final JSON files to stdout is important to your workflow, so that we can understand the use case and brainstorm possible solutions?

I want to get the rustdoc info about a workspace into my executable for further processing. Currently, I'm invoking this from a Rust host executable (via the rustdoc-json crate).

[weihanglo]

Thanks for sharing!

If I understand, it is more convenient and predictable if such a flag is provided, but not a hard requirement blocking your development, right?

In the meanwhile, I would suggest sticking with --message-format json, and parse the output. For example, filter in message of your targeted package_id, and get file paths from filenames.

{"reason":"compiler-artifact","package_id":"path+file:///projects/foo#0.1.0","manifest_path":"/projects/foo/Cargo.toml","target":{"kind":["lib"],"crate_types":["lib"],"name":"foo","src_path":"/projects/foo/src/lib.rs","edition":"2021","doc":true,"doctest":true,"test":true},"profile":{"opt_level":"0","debuginfo":2,"debug_assertions":true,"overflow_checks":true,"test":false},"features":[],"filenames":["/projects/foo/target/doc/foo.json"],"executable":null,"fresh":true}
{"reason":"build-finished","success":true}
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s
   Generated /projects/foo/target/doc/foo.json

You could also use the Generated status message, though that is subject to change and not for programming use.

[weihanglo]

whole rustdoc JSON output for a crate and all its deps to stdout instead of to a file

Note that flags specified in cargo rustdoc only applied to the rustdoc invocation of the final artifact, so it is unlikely to take effect on "all its deps".

I want to get the rustdoc info about a workspace into my executable for further processing.

It's not clear to me what kind of further processing it is. I'd like to mention it again that the current unstable --output-format doesn't handle dependencies. That is like a prerequisite to me for your use case, if we ever got this “output-to-stdout” support.