build icon indicating copy to clipboard operation
build copied to clipboard

Support clean JSON output of test result

Open eredo opened this issue 7 years ago • 4 comments

Previously it was possible to redirect the pub run test -r json output to a file and than parse the json file accordingly (ex. pub run test -r json > test_results.json). Now pub run build_runner test -- -r json cannot be redirected to a file because of the build output. A possible solution would be to provide an --output argument for build_runner test which pipes the test output to the file.

eredo avatar Jun 29 '18 10:06 eredo

Makes sense, we already have a --output flag but we could add a --test-output flag or similar (although that might be confusing as it would have a totally different meaning than --output).

Note that there is a workaround today, you can do pub run build_runner build test -o <output-dir>, and then pub run test -r json --precompiled <output-dir>.

Some other options could be:

  • Adding a flag to silence the build output entirely
  • Adding a json reporter mode to the build process as well

jakemac53 avatar Jun 29 '18 13:06 jakemac53

Long-er term I imagine you'll want a JSON reporter for other reasons (IDE integration, etc).

matanlurey avatar Jul 06 '18 17:07 matanlurey

We have some other ideas coming down the line which would be relevant as well when it comes to tool integration, so we might want to hold off on a full json reporter. Possibly something like this https://jvican.github.io/slides/BSP.pdf (or maybe just implement that with some language extensions).

jakemac53 avatar Jul 06 '18 17:07 jakemac53

I took a stab at fixing this issue here: https://github.com/dart-lang/build/pull/2508

I'm not sure if there are still plans for a full json reporter as mentioned above, but this would at least make it possible to get a usable json test report with a single build_runner command.

evanweible-wf avatar Oct 19 '19 23:10 evanweible-wf

The test package supports --file-reporter now, which I think can be used to accomplish this

jakemac53 avatar Sep 01 '23 18:09 jakemac53