CitrusSpringExtension for JUnit5 occasionally clears all Results

[tschlat]

Citrus Version 3.1.0

Expected behavior If I run more than one JUnitTest (Class) via CitrusSpringExtension, I would expect that the results of all TestCases over all TestClasses are collected and finally reported.

Actual behavior If some of the executed Tests differ in SpringContextConfiguration, more than one SpringContext is created during test executions (one for each configuration). For each SpringContextConfiguration the following instances (with the depicted dependencies) will be created:

• SpringContext
  • CitrusContext
    • TestReporters
      • TestResults

Now it is obvious that via this dependency chain TestResults are managed per SpringContext. Thus the actual result may differ from the expectation, because individual TestCases store their results in different TestResults instances.

The solution for this problem whould be to share one TestReporters instance over all SpringContexts. A respective bean could be stored in the root extension context and could be injected into new SpringContexts upon their creation.

There is another thing to mention. The current implementation of CitrusSpringExtension stores a Citrus instance in the root context. This storage implies that the Citrus instance should be used for all Tests executed by all Extensions. But the Citrus instance is replaced in the root context when a new SpringContext is created. This behaviour is certainly not thread safe and should probably also be corrected. -> Citrus instances should probably be stored per SpringContext.

Test case sample I will attach two test classes that reproduce this issue when they are executed as one suite.

If the depicted solution is OK I would implement it and provide a PR.

[tschlat]

Issue815TestSuite.zip

[christophd]

@tschlat yes this is a problem. Thank you for reporting and analyzing. We would be glad to have a PR for this

[tschlat]

Looks like this one has also been solved with this commit: https://github.com/citrusframework/citrus/commit/1252851a6cf08bd0d1024900fe0947da09dce5ad

@christophd could you please confirm that.

[christophd]

yes, should be solved now. Many thanks!