Support for running system tests without benchmarks and a database

[sema]

(1) We need to decide if we want non-pure HTML/JS benchmarks in our system tests.

Supporting PHP benchmarks by using the php command is okay, as it still renders our tests runnable out of the box. However, it is a hassle if the user needs to install mysql, set up permissions, and populate tables first.

I suggest that we move them into a sub-folder and disable them from running when issuing the python tests.py command.

(2) We should make it possible to run our system tests without the benchmarks, as the benchmarks requires a substantial time to finish (and we want to encourage developers to quickly run the system tests before they commit).

A --benchmarks and a --database-benchmarks parameter should be added to the test script, and only if these are given will the corresponding tests be executed.

[budde377]

How about changing the test suits to use SQLite or similar? This would allow us to provide a clean test db without all the configurations.

[sema]

That sounds like a good solution to the problem.