openaire
Concurrent builds on gorgona fail becaues of running out of memory
IIS tests on gorgona are sometimes killed (see eg. Crashed tests in https://gorgona.ocean.icm.edu.pl/job/openaire/job/iis/job/master/25/consoleFull), probably by the OOM killer. This happened recently during concurrent IIS builds, but may also happen when other jobs run concurrently with IIS builds. The default for -Xmx on Java 8 is 25% of memory, we can decrease it and be more memory friendly, but other concurrent builds can still cause problems.
Additionally it seems that in the current configuration crashed tests are detected only because of the interaction with the Sonar JaCoCo sensor - Sonar finds empty jacoco output and fails. The junit archiving plugin eg. does not see them, and with-Dmaven.test.failure.ignore apparently crashes are ignored too.
Reducing -Xmx is an option but from local testing it seems that the tests do not produce enough garbage to fill what they get by default anyway.
And today we got failures again when multiple builds of different branches were running concurrently.
To decrease pressure on the build machine we should probably start with adding disableConcurrentBuilds to the Jenkinsfile.