Aleksandar Prokopec

> Is the benchmark layer going to fork a VM? By default, the harness does not fork a new VM. The optional JMH-compliant build, however, uses JMH as the harness,...

> TLC fails if we check two different specs with a single VM instance (VM has to be fresh). In addition to what Francois said: you should be aware that...

@lemmy It's still possible to have single runs, but it would require setting the default number of repetitions to `1` in the benchmark definition. In this case, there would be...

I think that the `Context` object might need to include it.

Bibtex entry is not yet available from ACM DL, or can we already get it somewhere?

Perhaps we could additionally add a -v flag to enable verbose output that shows information such as this on stdout.

JSON sounds good to me then.

Odd, we added a full JMH build to the CI, which also runs all the benchmarks. We should check what went wrong (on the plane atm).

JMH build seemed to have passed in the CI: https://travis-ci.org/renaissance-benchmarks/renaissance/jobs/558938074#L537

Thanks for reporting this! Indeed, we should upgrade the Spark benchmarks. The current approach that we are planning is to open a new subproject that depends on a different Spark...