arquillian
We need some unit test implementation CI testing
Issue Overview
It seems like we need more unit test framework validation in the CI tests
Expected Behaviour
We should validate key versions of Junit 4/5 and TestNG
Current Behaviour
We don't have such tests
Not quite sure that'll be enough. Integration testing IMHO is the only surefire way to test things in this context. For example, how do you test Junit 5 extension integration points with a unit test?
I use https://github.com/flowlogix/test-arq-suite integration test for both JUnit 5 and TestNG canary tests, and so far this simple test translated 100% to all larger projects.
I need to do some more looking into this and if it can be shared across different organizations, but this looks interesting https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-apache-maven-registry. We might be able to deploy SNAPSHOT's there, then other projects can simply just use the repository to grab the SNAPSHOTS. This would allow projects to test against latest.
I do think we need some integration tests too. My initial thought is we create a integration-tests directory. We can have different modules for TestNG, JUnit and JUnit 5. Then we use profiles for different implementations.
Thinking a bit more about this. What would probably be best is to have a common module where all the tests live. Then, each TestNG, JUnit and JUnit 5 test suite can simply use the annotation required to enable the test. The one caveat here will be TestNG where, AFAIK, it's required to extend an abstract test.
The idea really being we don't want to have a ton of duplicated tests for each test framework.
Yes, we need an integration-test module that is skipped during the release but run for validation.
Looks like TestNG works. Couldn't really test JUnit but it looks ok. I think 1.9.0 release would be appreciated since 1.8.1 is broken :)
Thanks guys!