Disabling colored output requires convoluted `cargo test --color=never -- --color=never` incantation

[emidoots]

EDIT: see durka's comment below

It seems that there is no way to disable the colored output of cargo test. This is a nice thing to have when displaying output on e.g. CI sites that do not support ASCII escape codes.

Combinations I tried blindly that didn't work include:

• cargo test --no-color
• cargo test -- --no-color
• cargo test --color
• cargo test -- --color
• cargo test --color never

Also checked output of cargo -h and cargo test -h but saw nothing.

[durka]

On cargo 0.5.0 2015-09-04 (the one distributed with current nightly), the correct incantation seems to be

cargo test --color=never -- --color=never

Perhaps cargo test should automatically forward the color flag to the test runner?

[emidoots]

Thanks @durka! I can confirm that works. I will update the title of the issue because cargo test --color=never -- --color=never seems a bit convoluted to me :+1:

[alexcrichton]

While I think this should probably apply to all invocations of rustc or rustdoc, I'm much more wary to pass this down to test executables. Currently we have only one test framework but one day there will likely be many, and even today you can create test executables without a harness (e.g. no --test) where this option may not be supported or recognized.

I think a better solution may be for the test harness and Cargo to agree on an environment variable to disable or enable colorization or some other method not involving an extra argument.

[durka]

There is already RUST_TEST_THREADS, so following the pattern would be RUST_TEST_COLOR, I suppose?

[alexcrichton]

Sounds pretty reasonable to me!

[durka]

@alexcrichton though... were you thinking this env var would disable color in other Cargo modes, as well? Like building and doc-ing?

[durka]

The way I see it, there are two orthogonal-ish questions:

• Will the environment variable affect color just for testing or across all of the tooling (is rustc going to learn it)?
• Should cargo set this flag when passed --color or will that switch eventually go away leaving the environment variable as the Right Way to do it?

[alexcrichton]

So to start out with I was just thinking that whenever Cargo executed a test binary it would set this environment variable according to the flags passed to Cargo itself, but there's certainly a broader question of if there's an environment variable everyone can agree on to control this aspect. One has been proposed in https://github.com/rust-lang/rust/pull/27867, for example.

[stale[bot]]

As there hasn't been any activity here in a while would someone (the author, a team member, or any interested party) be able to summarise the current state, perhaps making explicit:

• Is this still relevant?
• If so, what is blocking it?
• Is it known what could be done to help move this forward?

Thank you!

(The cargo team is currently evaluating the use of Stale bot, and using #6035 as the tracking issue to gather feedback.)

If you're reading this comment from the distant future, fear not if this was closed automatically. If you believe it's still an issue please leave a comment and a team member can reopen this issue. Opening a new issue is also acceptable!

[stale[bot]]

As I didn't see any updates in 30 days I'm going to close this. Please see the previous comment for more information!

[ehuss]

I think this could probably stay open. I think there is an open question of forwarding flags to libtest, and how to handle other test harnesses (as being discussed in #6358 and #4325).

In particular, the following are also awkward:

cargo test -j2 -- --test-threads=2
cargo test --message-format=json -- --format=json
cargo test --quiet -- --quiet
cargo test --color=never -- --color=never

It might be a good idea to forward flags to libtest. But it would also be good to handle other harnesses, which isn't so straightforward. Environment variables are another option, which circumvents the issue of other harnesses not understanding the command-line (they would just be ignored).

[epage]

For #5609, I am looking at changing how cargo and libtest interact, with cargo taking more control over the rendering of output which would mean cargo will be showing the colored output, instead of libtest, resolving this issue.

[heaths]

There is already RUST_TEST_THREADS, so following the pattern would be RUST_TEST_COLOR, I suppose?

cargo build already respects the NO_COLOR env var e.g. NO_COLOR=1 cargo build so could that be supported as well? A lot of CLIs support NO_COLOR so it can be used pretty ubiquitously.

[epage]

Technically, cargo is not specified as supporting NO_COLOR but that is an accident of implementation.

The long term way of solving this is by having cargo in charge of rendering, like it is with rustc as I mentioned in #15343. The question is if we should do anything short term. For other reasons, I've been considering whether libtest should deprecate support for all environment variables, so it feels weird to add an environment variable when we may ~~drop support for~~ deprecate it soon after.

[heaths]

Given NO_COLOR is fairly ubiquitous, could cargo - especially once it handles all the rendering - document support for it? Env vars are handy when you want to control color uniformly across an entire command line expression. You don't need to pass something like --color=never to each command in the expression.

[epage]

CARGO_TERM_COLOR is our officially documented variable.

[epage]

I was not aware that Cargo currently conditionally forwards arguments to the test harness based on whether its a custom harness or not, see rust-lang/cargo#15465. https://github.com/rust-lang/cargo/blob/4b0e263feb62de7d9035171092e6b8448df73edf/src/cargo/ops/cargo_test.rs#L357-L359

I could see us forwarding

• term.color = "always" as --color=always
• term.color = "never" as --color=never

I'd hold off on forwarding --jobs <N> as --test-threads <N>. I lean towards not forwarding build.jobs but for us to instead have test.jobs and --jobs overwrites both. However, getting consensus on this might take time and distract from the other improvements.

We should not consider forwarding cargo test --message-format=json right now

• --format json is unstable
• This has greater implications for mixing of custom harnesses and built-in harness

The main set of questions I would have before doing any of this are:

• Do repeated arguments overwrite or conflict with themself? Say Cargo forwards --color=never and the user passes in --color=always, will that error or have the last one win?
• Is the repeated argument behavior considered specified behavior that we can rely on?

[epage]

Do repeated arguments overwrite or conflict with themself? Say Cargo forwards --color=never and the user passes in --color=always, will that error or have the last one win?

$ cargo test --test builder -- --color=never --color=always
     Running `/home/epage/.cargo/build/7e/902033f808610d/debug/deps/builder-ce06896cb4afb001 --color=never --color=alw
ays`
error: Option 'color' given more than once
error: test failed, to rerun pass `--test builder`

So this comes down to

Is the repeated argument behavior considered specified behavior that we can rely on?