Benchmark results website 2.0

[Akirathan]

Current solution (bad)

The current website with plots of all the benchmarks is a static website generated from a Jinja template by website_regen.py script hosted with GitHub pages on https://enso-org.github.io/engine-benchmark-results/.

This has many disadvantages:

• Hard to maintain. Especially with some Javascript code in the HTML template.
• The resulting web pages are huge and very slow to load, because they contain all the data inside Javascript embedded in the HTML.
• The Google charts JS library used by the template is ugly and difficult to work with and with insufficient documentation.
• It is very difficult to add any dynamic behavior like loading data on demand or comparing results of two branches.

Motivation

There are requests for new features in the current website:

• Plot also Enterprise edition benchmarks along with Community edition.
• Compare results from a custom branch to the results of the develop branch.
  • Currently handled only by the bench_download.py cmd line script.
• Display standard deviation.

It is easier to implement the current features along with the requested ones from scratch using VueJS than to add them to the current template solution.

VueJS proposal

Let's implement a single page application with VueJS and use some more responsive, sophisticated and better documented plotting library, like ChartJS. The data will be fetched from the cache directory in the same repo. We only need some index.json file that maps names of the files to their timestamps. There is no need for any sophisticated backend or a database yet. Just build static single page application with VueJS and host it on GitHub pages. Access to the cache directory in the same repo should be straightforward. The application won't be needed to regenerate on a daily basis, just when new features are added.

A prototype is being developed in https://github.com/enso-org/engine-benchmark-results/tree/vue

Benchmark data upload

The data for the benchmarks are uploaded daily by the Benchmarks Upload GH Action into the directory https://github.com/enso-org/engine-benchmark-results/tree/main/cache as JSON files. Note that this task does not need any change to the data uploading functionality.

Related

• https://github.com/enso-org/enso/issues/5714
• https://github.com/enso-org/enso/issues/9355