current-bench
current-bench copied to clipboard
ocurrent
Remote Runners
We're benchmarking Tezos on a Raspberry Pi for a number of reasons. It would be great if there was a notion of "remote runners" - install some software on the Pi that allows it to run the benchmarks, while the rest is running in the cloud somewhere.
Of course, if we go too far in this direction we've recreated other full-fledged CI systems out there (Jenkins, Circle CI, etc).
Thoughts?
The pipeline doesn't use it at the moment, but the ocurrent docker plugin allows you to specify a docker context for all the docker commands. See e.g.https://github.com/ocurrent/ocurrent/blob/master/plugins/docker/current_docker.mli#L35
We could extend the pipeline to support that. This context could then point to a remote server.
The remote host running the benchmarks would need to have relatively stable performance though (e.g. we use an isolated core on a bare metal host to run the irmin benchmarks). But I guess how sensitive the results are to this also depends on what you're testing.
We're doing pretty memory-sensitive benchmarks as well. I'd be interested to learn how exactly you set up the benchmarks for Irmin if you can share the details.
We're using a setup very similar to what is described here: https://github.com/ocurrent/current-bench#tuning-the-environment -- it's an intel-based server with an isolated core dedicated for benchmarks, with some additional adjustments such as power saving, interrupts and turbo boost disabled on the core. We're using a tmpfs mount for IO.