libtest-mimic icon indicating copy to clipboard operation
libtest-mimic copied to clipboard

JSON output

Open LukasKalbertodt opened this issue 7 years ago • 6 comments

Currently, --format=json is not supported. This should change.

LukasKalbertodt avatar Jul 23 '18 14:07 LukasKalbertodt

Any plans for this?

janosimas avatar Aug 15 '22 11:08 janosimas

I don't have plans to tackle this any time soon. What's your use case for it?

LukasKalbertodt avatar Aug 15 '22 11:08 LukasKalbertodt

I want to parse the output to use in a dashboard

janosimas avatar Aug 15 '22 11:08 janosimas

We should stick to the same format as the official test harness uses to allow drop-in replacement for tools that rely on it (like IDEs) - https://github.com/rust-lang/rust/issues/49359

Dinnerbone avatar Jan 27 '23 13:01 Dinnerbone

I just read the tracking issue for this feature again and it seems like libtest's JSON output is not really all that great. And certainly not close to stabilization from what I can tell.

So I worry that adding this to libtest-mimic lots of maintenance work as the output has to be adjusted to the latest nightly or whatever. On the other hand, the output doesn't seem to have changed a lot. So maybe it's fine.

LukasKalbertodt avatar Jan 29 '23 11:01 LukasKalbertodt

For reference, the output for tests/real/mixed_bag.rs is:

{ "type": "suite", "event": "started", "test_count": 8 }
{ "type": "test", "event": "started", "name": "blue" }
{ "type": "test", "event": "started", "name": "cat" }
{ "type": "test", "event": "started", "name": "cyan" }
{ "type": "test", "event": "started", "name": "dog" }
{ "type": "test", "event": "started", "name": "frog" }
{ "type": "test", "event": "started", "name": "owl" }
{ "type": "test", "event": "started", "name": "purple" }
{ "type": "test", "event": "started", "name": "red" }
{ "type": "test", "name": "cyan", "event": "ignored" }
{ "type": "test", "name": "cat", "event": "ok" }
{ "type": "test", "name": "frog", "event": "ignored" }
{ "type": "test", "name": "blue", "event": "failed", "stdout": "thread 'blue' panicked at 'sky fell down', mixed_bag.rs:32:5\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n" }
{ "type": "test", "name": "owl", "event": "ignored" }
{ "type": "test", "name": "purple", "event": "ignored" }
{ "type": "test", "name": "dog", "event": "failed", "stdout": "thread 'dog' panicked at 'was not a good boy', mixed_bag.rs:11:5\n" }
{ "type": "test", "name": "red", "event": "ok" }
{ "type": "suite", "event": "failed", "passed": 2, "failed": 2, "ignored": 4, "measured": 0, "filtered_out": 0, "exec_time": 0.050418943 }

LukasKalbertodt avatar Jan 29 '23 11:01 LukasKalbertodt