Separate unit tests from integration tests

[marcelstoer]

It's fantastic to have all the integration tests in src/test/java/e2e. The downside is that running them all is very time consuming. I would argue that having them executed when you run mvn test is unexpected.

It is Maven best/common practice to separate unit tests from integration tests.

• the unit tests are run by the surefire plugin in the test phase
• the integration tests are run by the failsafe plugin in the verify phase

By default integration test classes end in IT e.g. HelpIT instead of HelpTest.

So, either

• rename the classes in the e2e directory accordingly or
• configure the failsafe plugin such that it executes all tests inside that directory regardless of name

[danielflower]

Pretty much all the tests are e2e tests and I want them to always run so I'm not keen on doing this. I am keen on making them faster to run though, because it's painful how slow they are right now.

[marcelstoer]

The more a software component behaves differently from what one would expect based on common practice the better reasons you need to have 😉

I want them to always run

"Always" in what context? The Travis script will need an extra line as it runs only mvn test by default. If you are concerned about developers (i.e. you and PR contributors) not executing all tests before committing we could add <defaultGoal>verify</defaultGoal> to <build> in the pom.xml (then people would only have to type mvn or whatever).

it's painful how slow they are right now

It'll make sense to run them in parallel (e.g. forking 0.5 per CPU core), particularly once #91 is fixed.

[danielflower]

Well I tried running the tests concurrently. More than doubled the speed locally, but caused timeouts on travis-ci due to lack of stdout output during the build, so I reverted.