Automating parallel invocations of HsacFitNesseRunner?

[violinner]

As part of a Continuous-Integration build pipeline, I would like to automate running instances of HsacFitNesseRunner in parallel processes.

Is it possible to add the XML report formatter from standalone FitNesse to work with HsacFitNesseRunner?

I would like the results from each parallel execution to be collected by the test "robot" in the same way as other jobs using standalone-fitnesse.jar with the option:

-c MySuite?suite&format=xml&includehtml

[fhoeben]

You can probably create your own subclass of HsacFitNesseRunner to add that formatter, I suppose it is just a listener to be added.

For my curiosity: what is the advantage of the xml formatter over the 'surefire' xml format that is generated when running with HsacFitNesseRunner? In my experience all CI servers understand that format out-of-the-box whereas the FitNesse xml needs to be processed specially. In my experience running tests in parallel where the CI server just picks up each run's surefire xml and you combine the HTML results of all runs using HtmlReportIndexGenerator (which is then picked-up/published by the CI server) works like a charm with minimal effort. If you run in a containerised environment I especially recommend the docker images of https://github.com/fhoeben/hsac-fitnesse-docker, which also work great when splitting a test suite (and then creating a single overview report), but you probably already found those ;-)

[fhoeben]

Just looked up the SuiteXmlReformatter that generates the XML in the command line you suggested. It turns out that that is not a listener. From the first look of it it does not seem as straightforward as I hoped to add this.

[fhoeben]

Have you been able to address your need for parallel execution of tests on a CI server? What approach did you use?

[violinner]

CI test job (Jenkins) calls a BASH script with a parameterized list of sub-Suites to run in parallel. FitNesse is invoked once for each in the background (&), terminated by a "wait" command. An over-arching CI job timeout will stop all jobs if a given time-out is reached.

The Jenkins FitNesse plug-in collects all XML reports as "fitnesse-report*.xml", and all the independent jobs are reported, and fails the job if there are any errors in any of them.

This seems to work ok.

Caveats:

• All initialization must be done in the first Suite, marked "blocking" to complete before parallel suites begin.
• There are gigabytes of /dev/shm (shared memory) reserved for multiple Browser invocations.
• If java.awt.Robot is used (File uploads & downloads cannot be automated from Selenium; some other JavaScript interaction side-effects), multiple virtual DISPLAYs are needed.