gotestyourself
Append xml / json file when running more than 1 test suite in serial/parallel way
Hi,
Creating a new issue with the reference to the original reply post
I have a MAKE file that looks like below:
GOGETTESTSUM=go get gotest.tools/gotestsum
GOTESTSUM_JUNITFILE=gotestsum --junitfile unit-tests.xml
all: product1_region_uat product2_region_uat product3_region_uat
product1_region_uat:
$(GOGETTESTSUM)
parameters product1_test $(GOTESTSUM_JUNITFILE)
product2_region_uat:
$(GOGETTESTSUM)
parameters product2_test $(GOTESTSUM_JUNITFILE)
product3_region_uat:
$(GOGETTESTSUM)
parameters product3_test $(GOTESTSUM_JUNITFILE)
Each time I execute tests using make all to capture test outcome in unit-tests.xml file, result of only the last executed test suite is getting saved (i.e. product3 in this instance).
How can I save test execution for all 3 test suites?
Reply from @dnephin
Hello @viralkpanchal, thanks for your interest in gotestsum!
I think the problem you are seeing is that gotestsum is being run 3 times by the makefile. Every time gotestsum runs it will override the --jsonfile. I think you could fix the problem by having a target run all the packages at once, instead of one at a time, or you could use a different filename for each package. make has some fancy syntax for getting the name of the currently run target, which you could use as part of the jsonfile name. I don't remember exactly what that looks like.
Generally the idea is that running a single gotestsum invocation would be preferable. It is possible that --jsonfile could be made to append to the file, instead of overriding it. Or a new flag could be added to do append instead of override. I'm not sure if that is really the right solution, and would need to think about it some more. I think we could create a new issue for that change, as it is not exactly related to this problem described in this issue.
I hope that helps!
Originally posted by @dnephin in https://github.com/gotestyourself/gotestsum/issues/119#issuecomment-639615971
This problem came up again recently. Another workaround is to have each test run produce a jsonfile instead of a junitfile, and at the end run the command again to generate a single junitfile from all the json files. Something like this:
gotestsum --jsonfile=jsonfile-1 ...
gotestsum --jsonfile=jsonfile-2 ...
gotestsum --jsonfile=jsonfile-3 ...
gotestsum --junitfile=unit-tests.xml --raw-command | cat jsonfile-*
This works (without the |), however, it doesn't report the times from the test output, but only how long it took to process the JSON:
Example:
gotestsum --junitfile=unit-tests.xml --raw-command cat coverage/dashboard-all.json
<desired output>
DONE 1209 tests, 10 skipped in 0.019s
There are a bunch of options but none that I could surface to skip the DONE in output; | head -n -2 feels hacky.
