Make it possible to run two Renaissance instances side by side

[lbulej]

I'm not sure whether it is possible or not at the moment, but as we merge benchmarks with complex underlying frameworks, we should make sure that they can't step on each other's toes.

This is probably easy for things such as scratch space -- the harness should provide a benchmark with a scratch directory beneath a global scratch space.

It's probably going to be more difficult for things such as port numbers, if we use benchmarks that communicate over socket (Spark comes to mind).

[axel22]

That's a good point - we should probably create a scratch subdirectory when running a benchmark, and ensure that it runs within it.

Not sure how the address issues such as ports, other than being knowledgeable about this when writing the benchmark, and having the harness assign it through the configuration object.

[davleopo]

BTW in https://github.com/D-iii-S/renaissance-benchmarks/pull/17 I added logic to have common scratch dir generation for spark benchmarks, maybe we should adhere to this for the entire suite.

[axel22]

Regarding ports, note that many of the frameworks have some configurability options regarding the ports that they use.

[farquet]

Based on the non trivial aspect of the issue and the fact that there exists tools to trap processes in their own environment (firejail or even docker), I'm removing the 1.0.0 milestone for now.