Add unstable "--output-format" option to "doc" & "rustdoc"

[charmitro]

This commit enables the mimicking of --output-format option of "rustdoc".

We achieved this by:

• Handle --output-format argument, accepts "html" or "json".
• If "--ouput-format=json" we append the following to compile_opts.target_rustc_args:
  1. -Zunstable-commands
  2. --output-format

Part of #12103

[rustbot]

r? @ehuss

(rustbot has picked a reviewer for you, use r? to override)

[charmitro]

I'm a little confused about the -Zunstable-options etc. Any comment on that will help.

[charmitro]

Thanks for the contribution!

Haven't read through the entire change. Just wondering if you could add tests that verify without the -Z flag Cargo will fail, for both rustdoc and doc commands.

Thanks! Added a couple more.

[ehuss]

r? @weihanglo

It looks like you have this covered, but feel free to reassign it back to me if you want.

[charmitro]

Fingerprint detection for <crate>.json

#12103 is an issue originated from #12100. The most important line in description of #12103 is missing in the current patch:

This would allow cargo to reason about it and take it into account in the fingerprinting logic.

If we look into #12100 carefully, we shall the change touched this piece of code:

https://github.com/rust-lang/cargo/blob/928b9561c16e73fca6206b42e4c464b80b0e9221/src/cargo/core/compiler/context/compilation_files.rs#L439-L442

This is the core logic Cargo determines where final output artifacts are located. We should do some trict here in order to make the fingerprint (the re-compile detection) takes <crate>.json into account.

In order to do so, Unit or some other types involved in the above code may need to know about --output-format. This is one non-easy part.

Is it safe to say that we should add a, e.g. json: bool or output_format: OutputFormat in CompileMode::Doc so we can determine whether the fingerprint should look for index.html or <crate>.json?

[weihanglo]

Is it safe to say that we should add a, e.g. json: bool or output_format: OutputFormat in CompileMode::Doc so we can determine whether the fingerprint should look for index.html or <crate>.json?

It's really tempting to do that! I feel a bit uncertain to add more fields onto CompileMode::Doc though I cannot find any better place than that. CompileMode is also already tracked in the fingerprint, so recompile should be automatically detected. I think it worth giving an attempt.

[weihanglo]

@ehuss do you have any input on adding a new field on CompileMode::Doc? Is there any better alternative?

[ehuss]

CompileMode::Doc seems like the correct place, especially with output computation and fingerprinting being involved. I would also expect things like calc_outputs to get updated.

[charmitro]

CompileMode::Doc seems like the correct place, especially with output computation and fingerprinting being involved. I would also expect things like calc_outputs to get updated.

Ack.

I will move forward with this and the rest of the requested changes and ping back. I will try to focus on cargo rustdoc for now and if the waters are calm I will also propose the cargo doc also.

[charmitro]

Patch v2 updates:

• Change now affects only cargo rustdoc, restoring cargo doc to it's current upstream state(rebased)
• Fingerprint detection of <crate>.json in the case we are building documentation with output-format=json
• Derive cleanup of enum OutputFormat & refactor .eq to matches!
• Use fail_if_stable_opt where needed

If some requested changes are not resolved yet, will updated the patch again.

How can I test that the fingerprint works? Is there a reference for such tests? I found many containing fingerprint related code, but it seems somewhat delicate to poke around with no previous experience on the matter.

[bors]

:umbrella: The latest upstream changes (presumably #11905) made this pull request unmergeable. Please resolve the merge conflicts.

[weihanglo]

Ping @charmitro. Just checking in to see if you are still interested in working on this, or if you had any questions.

@rustbot author

[charmitro]

Ping @charmitro. Just checking in to see if you are still interested in working on this, or if you had any questions.

@rustbot author

Still interested but currently and until start of September I'm on vacation. I'll be able to continue work afterwards.

My bad for not mentioning earlier that I'm AFK. Thanks for asking!

[charmitro]

@weihanglo FYI I'm starting on it again. I'll have this PR updated soon.

[bors]

:umbrella: The latest upstream changes (presumably #13168) made this pull request unmergeable. Please resolve the merge conflicts.

[charmitro]

PR updated. All comments were addressed.

A major change in this revision is allowing --open with --output-format json.

[charmitro]

Thanks for the update!

There are some nits and unresolved comments, and then we're ready to go!

* [Add unstable `--output-format` option to  `cargo rustdoc` #12252 (comment)](https://github.com/rust-lang/cargo/pull/12252#discussion_r1448324026)

* [Add unstable `--output-format` option to  `cargo rustdoc` #12252 (comment)](https://github.com/rust-lang/cargo/pull/12252#discussion_r1448324634)

Resolved!

[weihanglo]

Great job!

Thanks a lot for the collaboration :)

@bors r+

[bors]

:pushpin: Commit 9276399342c32b87a0126e3ef88864c6aa1bc576 has been approved by weihanglo

It is now in the queue for this repository.

[weihanglo]

@charmitro

Could you add a new doc entry of https://doc.rust-lang.org/nightly/cargo/reference/unstable.html for this feature?

You can take this as a reference: https://github.com/rust-lang/cargo/pull/10383/commits/4e45f58852313e3dde5f617e2c26bbda5e9437c1#diff-e35cef716988e9f7122a9c90479aa9204e61d1f41b094c0d183a44e0ca271eaaR83-R469

I've created the tracking issue here https://github.com/rust-lang/cargo/issues/13283

[charmitro]

@charmitro

Could you add a new doc entry of https://doc.rust-lang.org/nightly/cargo/reference/unstable.html for this feature?

You can take this as a reference: 4e45f58#diff-e35cef716988e9f7122a9c90479aa9204e61d1f41b094c0d183a44e0ca271eaaR83-R469

I've created the tracking issue here #13283

In this PR or create a follow-up?

[weihanglo]

This PR is now in the bors merge queue, so a follow-up would be better :)

[bors]

:hourglass: Testing commit 9276399342c32b87a0126e3ef88864c6aa1bc576 with merge f4e2eac630b06bf13906ea609f2b42cb15600d7d...

[bors]

:sunny: Test successful - checks-actions Approved by: weihanglo Pushing f4e2eac630b06bf13906ea609f2b42cb15600d7d to master...